2024-09-15T21:03:51.623 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:03:51.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:03:51.725 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674 branch: main description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zlib} rados supported-random-distro$/{ubuntu_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905674' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: global: mon client directed command retry: 5 mon client hunt interval max multiple: 2 mon election default strategy: 3 mon mgr beacon grace: 90 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 1000 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNBz+BJB5K9tmFCvufx3TPBjyTqn4Umkn+kchncuxTpuiClZUxA6kHujRDfrJNZ0BrvOP3Rabzvoi3HAn8qjjI= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:03:51.725 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T21:03:51.726 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T21:03:51.726 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:03:51.728 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:03:51.749 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T21:03:51.749 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:03:51.749 INFO:teuthology.packaging:ref: None 2024-09-15T21:03:51.749 INFO:teuthology.packaging:tag: None 2024-09-15T21:03:51.750 INFO:teuthology.packaging:branch: main 2024-09-15T21:03:51.750 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:03:51.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-15T21:03:51.958 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:03:51.959 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:03:51.965 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:03:51.965 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:03:51.971 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:03:51.982 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:03:51.992 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:03:52.016 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 20:58:22.101656', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF8EAYqdtpaCg2TU0WtJWk7Ain/KvEywK3yLQNesCzEq'} 2024-09-15T21:03:52.016 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:03:52.022 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-15T21:03:52.022 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:03:52.059 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fce1bdeb010>, signals=[15]) 2024-09-15T21:03:52.059 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:03:52.063 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:03:52.064 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:03:52.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:03:52.148 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:03:52.154 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-15T21:03:52.253 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-15T21:03:52.254 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-15T21:03:52.301 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-15T21:03:52.301 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Ubuntu" 2024-09-15T21:03:52.301 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="22.04" 2024-09-15T21:03:52.301 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_CODENAME=jammy 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:ID=ubuntu 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE=debian 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T21:03:52.302 INFO:teuthology.orchestra.run.smithi114.stdout:UBUNTU_CODENAME=jammy 2024-09-15T21:03:52.303 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-15T21:03:52.320 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:03:52.328 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:03:52.334 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:03:52.334 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:03:52.348 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:03:52.355 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:03:52.355 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:03:52.398 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:03:52.432 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:03:52.447 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:03:52.447 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T21:03:52.448 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:03:52.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:03:52.453 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-15T21:03:52.505 INFO:teuthology.orchestra.run.smithi114.stdout:5.15.0-56-generic 2024-09-15T21:03:52.505 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.15.0-56-generic 2024-09-15T21:03:52.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2024-09-15T21:03:52.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-15T21:03:52.928 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:03:52.930 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-15T21:03:52.981 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-15T21:03:53.337 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-15T21:03:53.361 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-15T21:03:53.477 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-15T21:03:53.657 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-15T21:03:53.794 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-15T21:03:53.831 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-15T21:03:53.832 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-15T21:03:53.843 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-15T21:03:53.844 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-15T21:03:53.844 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-15T21:03:53.845 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-15T21:03:53.847 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-15T21:03:53.917 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-15T21:03:53.929 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-15T21:03:53.953 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-15T21:03:53.953 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-15T21:03:53.953 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-15T21:03:53.963 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-15T21:03:53.963 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-15T21:03:53.971 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-15T21:03:53.983 INFO:teuthology.orchestra.run.smithi114.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-15T21:03:53.989 INFO:teuthology.orchestra.run.smithi114.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-15T21:03:53.991 INFO:teuthology.orchestra.run.smithi114.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-15T21:03:53.992 INFO:teuthology.orchestra.run.smithi114.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-15T21:03:53.992 INFO:teuthology.orchestra.run.smithi114.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-15T21:03:53.993 INFO:teuthology.orchestra.run.smithi114.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-15T21:03:53.993 INFO:teuthology.orchestra.run.smithi114.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-15T21:03:53.994 INFO:teuthology.orchestra.run.smithi114.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-15T21:03:54.001 INFO:teuthology.orchestra.run.smithi114.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-15T21:03:54.001 INFO:teuthology.orchestra.run.smithi114.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-15T21:03:54.040 INFO:teuthology.orchestra.run.smithi114.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-15T21:03:54.075 INFO:teuthology.orchestra.run.smithi114.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-15T21:03:54.076 INFO:teuthology.orchestra.run.smithi114.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-15T21:03:54.077 INFO:teuthology.orchestra.run.smithi114.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-15T21:03:54.077 INFO:teuthology.orchestra.run.smithi114.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-15T21:03:54.077 INFO:teuthology.orchestra.run.smithi114.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-15T21:03:54.082 INFO:teuthology.orchestra.run.smithi114.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-15T21:03:54.082 INFO:teuthology.orchestra.run.smithi114.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-15T21:03:54.121 INFO:teuthology.orchestra.run.smithi114.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-15T21:03:54.122 INFO:teuthology.orchestra.run.smithi114.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-15T21:03:54.122 INFO:teuthology.orchestra.run.smithi114.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-15T21:04:00.038 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 16.0 MB in 3s (5,783 kB/s) 2024-09-15T21:04:01.516 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:04:01.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:01.591 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:04:01.777 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-15T21:04:01.777 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-15T21:04:01.943 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:01.943 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:01.943 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:01.943 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:01.944 INFO:teuthology.orchestra.run.smithi114.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-15T21:04:01.944 INFO:teuthology.orchestra.run.smithi114.stdout: python2.7-minimal 2024-09-15T21:04:01.944 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:01.945 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2024-09-15T21:04:01.945 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:01.945 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-15T21:04:01.945 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:01.946 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2024-09-15T21:04:01.947 INFO:teuthology.orchestra.run.smithi114.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-15T21:04:01.947 INFO:teuthology.orchestra.run.smithi114.stdout:Recommended packages: 2024-09-15T21:04:01.947 INFO:teuthology.orchestra.run.smithi114.stdout: thermald 2024-09-15T21:04:02.010 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2024-09-15T21:04:02.010 INFO:teuthology.orchestra.run.smithi114.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-15T21:04:02.010 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-15T21:04:02.010 INFO:teuthology.orchestra.run.smithi114.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-15T21:04:02.012 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages will be upgraded: 2024-09-15T21:04:02.012 INFO:teuthology.orchestra.run.smithi114.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T21:04:02.191 INFO:teuthology.orchestra.run.smithi114.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:02.192 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 113 MB of archives. 2024-09-15T21:04:02.192 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-15T21:04:02.192 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-15T21:04:03.586 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-15T21:04:03.940 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-15T21:04:06.114 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-15T21:04:06.114 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-15T21:04:06.114 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-15T21:04:06.520 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-15T21:04:06.571 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-15T21:04:06.949 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 113 MB in 5s (24.9 MB/s) 2024-09-15T21:04:07.169 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-15T21:04:09.183 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-15T21:04:09.187 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:09.316 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:11.885 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-15T21:04:11.905 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:11.996 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:12.531 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-15T21:04:12.552 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:12.595 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:19.181 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:19.305 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:19.653 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:19.792 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:20.103 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-15T21:04:20.114 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-15T21:04:20.144 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:26.386 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-15T21:04:26.424 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-15T21:04:26.461 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:30.408 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-15T21:04:30.522 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-15T21:04:30.856 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-15T21:04:31.002 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:31.153 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-15T21:04:31.281 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:32.639 INFO:teuthology.orchestra.run.smithi114.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:32.639 INFO:teuthology.orchestra.run.smithi114.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:32.639 INFO:teuthology.orchestra.run.smithi114.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:32.639 INFO:teuthology.orchestra.run.smithi114.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:32.809 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:34.236 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:35.711 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-15T21:04:35.837 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-15T21:04:35.988 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-15T21:04:36.049 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T21:04:36.049 INFO:teuthology.orchestra.run.smithi114.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:50.218 INFO:teuthology.orchestra.run.smithi114.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T21:04:50.219 INFO:teuthology.orchestra.run.smithi114.stdout:Sourcing file `/etc/default/grub' 2024-09-15T21:04:50.247 INFO:teuthology.orchestra.run.smithi114.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:50.273 INFO:teuthology.orchestra.run.smithi114.stdout:Generating grub configuration file ... 2024-09-15T21:04:50.586 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:50.609 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:50.972 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:50.978 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:51.049 INFO:teuthology.orchestra.run.smithi114.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:51.054 INFO:teuthology.orchestra.run.smithi114.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:51.215 INFO:teuthology.orchestra.run.smithi114.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:51.215 INFO:teuthology.orchestra.run.smithi114.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:51.216 INFO:teuthology.orchestra.run.smithi114.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:51.263 INFO:teuthology.orchestra.run.smithi114.stdout:done 2024-09-15T21:04:51.790 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2024-09-15T21:04:51.818 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2024-09-15T21:04:51.818 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2024-09-15T21:04:51.818 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2024-09-15T21:04:51.818 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2024-09-15T21:04:51.818 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 21 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:51.819 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2024-09-15T21:04:51.820 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-15T21:04:51.820 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:04:51.820 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:04:51.820 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T21:04:51.820 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:04:51.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get clean 2024-09-15T21:04:52.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-15T21:04:52.272 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:04:52.272 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:04:52.343 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:04:52.424 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:04:53.692 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:04:53.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T21:04:53.773 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:04:53.984 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-15T21:04:53.985 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-15T21:04:54.166 INFO:teuthology.orchestra.run.smithi114.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-15T21:04:54.166 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:04:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:04:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:04:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:04:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:04:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:04:54.167 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:04:54.206 INFO:teuthology.orchestra.run.smithi114.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-15T21:04:54.208 DEBUG:teuthology.orchestra.run.smithi114:> dpkg -s linux-image-generic 2024-09-15T21:04:54.220 INFO:teuthology.orchestra.run.smithi114.stdout:Package: linux-image-generic 2024-09-15T21:04:54.220 INFO:teuthology.orchestra.run.smithi114.stdout:Status: install ok installed 2024-09-15T21:04:54.220 INFO:teuthology.orchestra.run.smithi114.stdout:Priority: optional 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Section: kernel 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Installed-Size: 21 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: amd64 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Source: linux-meta 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Version: 5.15.0.121.121 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T21:04:54.221 INFO:teuthology.orchestra.run.smithi114.stdout:Recommends: thermald 2024-09-15T21:04:54.222 INFO:teuthology.orchestra.run.smithi114.stdout:Description: Generic Linux kernel image 2024-09-15T21:04:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: This package will always depend on the latest generic kernel image 2024-09-15T21:04:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: available. 2024-09-15T21:04:54.222 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-15T21:04:54.267 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.V3Rwy3MKBa 2024-09-15T21:04:54.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub/grub.cfg /tmp/tmp.V3Rwy3MKBa 2024-09-15T21:04:54.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.V3Rwy3MKBa 2024-09-15T21:04:54.446 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.V3Rwy3MKBa is 10KB 2024-09-15T21:04:54.459 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.V3Rwy3MKBa 2024-09-15T21:04:54.465 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:54.520 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:04:54.520 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T21:04:54.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T21:04:54.599 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-15T21:04:54.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo update-grub 2024-09-15T21:04:56.023 INFO:teuthology.orchestra.run.smithi114.stderr:Sourcing file `/etc/default/grub' 2024-09-15T21:04:56.024 INFO:teuthology.orchestra.run.smithi114.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T21:04:56.035 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-15T21:04:56.235 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-15T21:04:56.242 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-15T21:04:56.522 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-15T21:04:56.527 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-15T21:04:56.572 INFO:teuthology.orchestra.run.smithi114.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-15T21:04:56.576 INFO:teuthology.orchestra.run.smithi114.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-15T21:04:56.748 INFO:teuthology.orchestra.run.smithi114.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-15T21:04:56.748 INFO:teuthology.orchestra.run.smithi114.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-15T21:04:56.748 INFO:teuthology.orchestra.run.smithi114.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-15T21:04:56.759 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-15T21:04:56.761 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-15T21:05:26.794 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T21:05:26.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:05:26.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:45.281 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-15T21:05:54.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:05:54.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:05:57.345 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-15T21:06:09.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:06:09.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:06:27.773 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-15T21:06:42.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:06:42.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:06:43.432 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-15T21:06:44.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:06:44.190 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-15T21:06:44.191 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-15T21:06:44.237 INFO:teuthology.orchestra.run.smithi114.stdout:5.15.0-121-generic 2024-09-15T21:06:44.238 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-15T21:06:44.238 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:06:44.238 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T21:06:45.239 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T21:06:45.239 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:06:45.354 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-15T21:06:45.391 DEBUG:teuthology.parallel:result is None 2024-09-15T21:06:45.391 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:06:45.399 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:06:45.399 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:06:45.404 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:06:45.467 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:06:45.473 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:06:45.473 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:06:45.488 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:06:45.495 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:06:45.495 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:06:45.550 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:06:45.558 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:06:45.559 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:06:45.566 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:06:45.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:06:45.628 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:06:45.637 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:06:45.637 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:06:45.674 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:06:45.722 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:06:45.770 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:06:45.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:06:45.834 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-15T21:06:45.919 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:06:45.926 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:06:45.926 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:06:45.937 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:06:45.946 DEBUG:teuthology.task.selinux:Excluding smithi114: OS 'ubuntu' does not support SELinux 2024-09-15T21:06:45.946 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:06:45.946 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T21:06:45.946 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:06:45.946 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:06:45.957 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:06:45.968 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:06:45.969 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:14:14.528 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-09-15T21:14:14.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:14:14.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:14:14.617 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-15T21:14:14.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-15T21:14:14.689 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:14:14.699 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:14:14.699 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:14:14.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:14:14.757 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-15T21:14:14.757 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Command line: ntpd -gq 2024-09-15T21:14:14.757 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: ---------------------------------------------------- 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: ntp-4 is maintained by Network Time Foundation, 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: corporation. Support and training for ntp-4 are 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: available at https://www.nwtime.org/support 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: ---------------------------------------------------- 2024-09-15T21:14:14.758 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: proto: precision = 0.049 usec (-24) 2024-09-15T21:14:14.759 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: basedate set to 2022-02-04 2024-09-15T21:14:14.759 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: gps base set to 2022-02-06 (week 2196) 2024-09-15T21:14:14.759 INFO:teuthology.orchestra.run.smithi114.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:14.759 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T21:14:14.759 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen normally on 3 enp3s0f1 172.21.15.114:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen normally on 4 lo [::1]:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cb63%5]:123 2024-09-15T21:14:14.761 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:14 ntpd[18488]: Listening on routing socket on fd #22 for interface updates 2024-09-15T21:14:15.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:15.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:15.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:15.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:15.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:15.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:15.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:15.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:15 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:17.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:17 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:18 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:19.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:19.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:19.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:19.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:19.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:19.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:19 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:20.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:20.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:20.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:20.758 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:20.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:20.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:20 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T21:14:21.759 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T21:14:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:15 Sep 21:14:21 ntpd[18488]: ntpd: time slew -0.002344 s 2024-09-15T21:14:21.760 INFO:teuthology.orchestra.run.smithi114.stdout:ntpd: time slew -0.002344s 2024-09-15T21:14:21.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T21:14:21.760 INFO:teuthology.orchestra.run.smithi114.stderr:15 Sep 21:14:21 ntpd[18488]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================== 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-15T21:14:21.809 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:14:21.820 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:14:21.820 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T21:14:21.821 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T21:14:21.821 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:14:21.826 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:14:21.826 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:14:21.827 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-key list | grep Ceph 2024-09-15T21:14:21.909 INFO:teuthology.orchestra.run.smithi114.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-15T21:14:21.937 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T21:14:21.937 INFO:teuthology.orchestra.run.smithi114.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T21:14:21.937 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T21:14:21.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:14:22.081 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default/ 2024-09-15T21:14:22.081 INFO:teuthology.task.install.deb:Package version is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:22.191 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:14:22.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T21:14:22.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo apt-get update 2024-09-15T21:14:22.426 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy InRelease 2024-09-15T21:14:22.447 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-15T21:14:22.471 INFO:teuthology.orchestra.run.smithi114.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-15T21:14:22.492 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-15T21:14:22.511 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-15T21:14:22.512 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-15T21:14:22.532 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,427 B] 2024-09-15T21:14:22.575 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-15T21:14:22.657 INFO:teuthology.orchestra.run.smithi114.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-15T21:14:22.884 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 29.2 kB in 0s (59.0 kB/s) 2024-09-15T21:14:23.916 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:14:23.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4930-gfc3e0bff-1jammy cephadm=19.3.0-4930-gfc3e0bff-1jammy ceph-mds=19.3.0-4930-gfc3e0bff-1jammy ceph-mgr=19.3.0-4930-gfc3e0bff-1jammy ceph-common=19.3.0-4930-gfc3e0bff-1jammy ceph-fuse=19.3.0-4930-gfc3e0bff-1jammy ceph-test=19.3.0-4930-gfc3e0bff-1jammy ceph-volume=19.3.0-4930-gfc3e0bff-1jammy radosgw=19.3.0-4930-gfc3e0bff-1jammy python3-rados=19.3.0-4930-gfc3e0bff-1jammy python3-rgw=19.3.0-4930-gfc3e0bff-1jammy python3-cephfs=19.3.0-4930-gfc3e0bff-1jammy python3-rbd=19.3.0-4930-gfc3e0bff-1jammy libcephfs2=19.3.0-4930-gfc3e0bff-1jammy libcephfs-dev=19.3.0-4930-gfc3e0bff-1jammy librados2=19.3.0-4930-gfc3e0bff-1jammy librbd1=19.3.0-4930-gfc3e0bff-1jammy rbd-fuse=19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:14:23.994 INFO:teuthology.orchestra.run.smithi114.stdout:Reading package lists... 2024-09-15T21:14:24.203 INFO:teuthology.orchestra.run.smithi114.stdout:Building dependency tree... 2024-09-15T21:14:24.204 INFO:teuthology.orchestra.run.smithi114.stdout:Reading state information... 2024-09-15T21:14:24.399 INFO:teuthology.orchestra.run.smithi114.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T21:14:24.399 INFO:teuthology.orchestra.run.smithi114.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-15T21:14:24.399 INFO:teuthology.orchestra.run.smithi114.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-15T21:14:24.399 INFO:teuthology.orchestra.run.smithi114.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-15T21:14:24.400 INFO:teuthology.orchestra.run.smithi114.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-15T21:14:24.400 INFO:teuthology.orchestra.run.smithi114.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-15T21:14:24.400 INFO:teuthology.orchestra.run.smithi114.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T21:14:24.400 INFO:teuthology.orchestra.run.smithi114.stdout:The following additional packages will be installed: 2024-09-15T21:14:24.400 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T21:14:24.401 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-15T21:14:24.401 INFO:teuthology.orchestra.run.smithi114.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-15T21:14:24.401 INFO:teuthology.orchestra.run.smithi114.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-15T21:14:24.401 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-15T21:14:24.402 INFO:teuthology.orchestra.run.smithi114.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-15T21:14:24.403 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet zip 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout:Suggested packages: 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout:Recommended packages: 2024-09-15T21:14:24.404 INFO:teuthology.orchestra.run.smithi114.stdout: btrfs-tools 2024-09-15T21:14:24.507 INFO:teuthology.orchestra.run.smithi114.stdout:The following NEW packages will be installed: 2024-09-15T21:14:24.507 INFO:teuthology.orchestra.run.smithi114.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T21:14:24.507 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T21:14:24.507 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-15T21:14:24.508 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-15T21:14:24.508 INFO:teuthology.orchestra.run.smithi114.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-15T21:14:24.508 INFO:teuthology.orchestra.run.smithi114.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-15T21:14:24.509 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-15T21:14:24.510 INFO:teuthology.orchestra.run.smithi114.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-15T21:14:24.568 INFO:teuthology.orchestra.run.smithi114.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-15T21:14:24.568 INFO:teuthology.orchestra.run.smithi114.stdout:Need to get 244 MB of archives. 2024-09-15T21:14:24.568 INFO:teuthology.orchestra.run.smithi114.stdout:After this operation, 909 MB of additional disk space will be used. 2024-09-15T21:14:24.568 INFO:teuthology.orchestra.run.smithi114.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-15T21:14:24.651 INFO:teuthology.orchestra.run.smithi114.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,633 kB] 2024-09-15T21:14:24.658 INFO:teuthology.orchestra.run.smithi114.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-15T21:14:24.661 INFO:teuthology.orchestra.run.smithi114.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-15T21:14:24.678 INFO:teuthology.orchestra.run.smithi114.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-15T21:14:24.737 INFO:teuthology.orchestra.run.smithi114.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-15T21:14:24.741 INFO:teuthology.orchestra.run.smithi114.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-15T21:14:24.754 INFO:teuthology.orchestra.run.smithi114.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-15T21:14:24.757 INFO:teuthology.orchestra.run.smithi114.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-15T21:14:24.759 INFO:teuthology.orchestra.run.smithi114.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-15T21:14:24.759 INFO:teuthology.orchestra.run.smithi114.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-15T21:14:24.760 INFO:teuthology.orchestra.run.smithi114.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-15T21:14:24.768 INFO:teuthology.orchestra.run.smithi114.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-15T21:14:24.771 INFO:teuthology.orchestra.run.smithi114.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-15T21:14:24.773 INFO:teuthology.orchestra.run.smithi114.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-15T21:14:24.774 INFO:teuthology.orchestra.run.smithi114.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-15T21:14:24.775 INFO:teuthology.orchestra.run.smithi114.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-15T21:14:24.792 INFO:teuthology.orchestra.run.smithi114.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-15T21:14:24.794 INFO:teuthology.orchestra.run.smithi114.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-15T21:14:24.796 INFO:teuthology.orchestra.run.smithi114.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-15T21:14:24.797 INFO:teuthology.orchestra.run.smithi114.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-15T21:14:24.797 INFO:teuthology.orchestra.run.smithi114.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-15T21:14:24.799 INFO:teuthology.orchestra.run.smithi114.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-15T21:14:24.799 INFO:teuthology.orchestra.run.smithi114.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-15T21:14:24.799 INFO:teuthology.orchestra.run.smithi114.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-15T21:14:24.809 INFO:teuthology.orchestra.run.smithi114.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-15T21:14:24.809 INFO:teuthology.orchestra.run.smithi114.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-15T21:14:24.827 INFO:teuthology.orchestra.run.smithi114.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-15T21:14:24.827 INFO:teuthology.orchestra.run.smithi114.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-15T21:14:24.829 INFO:teuthology.orchestra.run.smithi114.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-15T21:14:24.830 INFO:teuthology.orchestra.run.smithi114.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-15T21:14:24.830 INFO:teuthology.orchestra.run.smithi114.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-15T21:14:24.831 INFO:teuthology.orchestra.run.smithi114.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-15T21:14:24.831 INFO:teuthology.orchestra.run.smithi114.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T21:14:24.832 INFO:teuthology.orchestra.run.smithi114.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-15T21:14:24.835 INFO:teuthology.orchestra.run.smithi114.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4930-gfc3e0bff-1jammy [3,334 kB] 2024-09-15T21:14:24.843 INFO:teuthology.orchestra.run.smithi114.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-15T21:14:24.844 INFO:teuthology.orchestra.run.smithi114.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-15T21:14:24.860 INFO:teuthology.orchestra.run.smithi114.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-15T21:14:24.861 INFO:teuthology.orchestra.run.smithi114.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-15T21:14:24.866 INFO:teuthology.orchestra.run.smithi114.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-15T21:14:24.866 INFO:teuthology.orchestra.run.smithi114.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-15T21:14:24.867 INFO:teuthology.orchestra.run.smithi114.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-15T21:14:24.867 INFO:teuthology.orchestra.run.smithi114.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-15T21:14:24.869 INFO:teuthology.orchestra.run.smithi114.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-15T21:14:24.871 INFO:teuthology.orchestra.run.smithi114.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-15T21:14:24.877 INFO:teuthology.orchestra.run.smithi114.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-15T21:14:24.879 INFO:teuthology.orchestra.run.smithi114.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-15T21:14:24.881 INFO:teuthology.orchestra.run.smithi114.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4930-gfc3e0bff-1jammy [721 kB] 2024-09-15T21:14:24.891 INFO:teuthology.orchestra.run.smithi114.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-15T21:14:24.893 INFO:teuthology.orchestra.run.smithi114.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4930-gfc3e0bff-1jammy [356 kB] 2024-09-15T21:14:24.893 INFO:teuthology.orchestra.run.smithi114.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-15T21:14:24.894 INFO:teuthology.orchestra.run.smithi114.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-15T21:14:24.894 INFO:teuthology.orchestra.run.smithi114.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-15T21:14:24.897 INFO:teuthology.orchestra.run.smithi114.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-15T21:14:24.899 INFO:teuthology.orchestra.run.smithi114.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4930-gfc3e0bff-1jammy [32.8 kB] 2024-09-15T21:14:24.899 INFO:teuthology.orchestra.run.smithi114.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4930-gfc3e0bff-1jammy [180 kB] 2024-09-15T21:14:24.902 INFO:teuthology.orchestra.run.smithi114.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4930-gfc3e0bff-1jammy [71.3 kB] 2024-09-15T21:14:24.903 INFO:teuthology.orchestra.run.smithi114.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4930-gfc3e0bff-1jammy [338 kB] 2024-09-15T21:14:24.909 INFO:teuthology.orchestra.run.smithi114.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4930-gfc3e0bff-1jammy [7,966 kB] 2024-09-15T21:14:24.950 INFO:teuthology.orchestra.run.smithi114.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-15T21:14:24.950 INFO:teuthology.orchestra.run.smithi114.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-15T21:14:24.951 INFO:teuthology.orchestra.run.smithi114.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T21:14:24.952 INFO:teuthology.orchestra.run.smithi114.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-15T21:14:24.953 INFO:teuthology.orchestra.run.smithi114.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-15T21:14:24.970 INFO:teuthology.orchestra.run.smithi114.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-15T21:14:24.972 INFO:teuthology.orchestra.run.smithi114.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-15T21:14:24.972 INFO:teuthology.orchestra.run.smithi114.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-15T21:14:24.987 INFO:teuthology.orchestra.run.smithi114.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-15T21:14:24.987 INFO:teuthology.orchestra.run.smithi114.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-15T21:14:24.988 INFO:teuthology.orchestra.run.smithi114.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-15T21:14:24.988 INFO:teuthology.orchestra.run.smithi114.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-15T21:14:24.988 INFO:teuthology.orchestra.run.smithi114.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-15T21:14:24.989 INFO:teuthology.orchestra.run.smithi114.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-15T21:14:25.003 INFO:teuthology.orchestra.run.smithi114.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-15T21:14:25.003 INFO:teuthology.orchestra.run.smithi114.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-15T21:14:25.006 INFO:teuthology.orchestra.run.smithi114.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-15T21:14:25.020 INFO:teuthology.orchestra.run.smithi114.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-15T21:14:25.021 INFO:teuthology.orchestra.run.smithi114.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-15T21:14:25.021 INFO:teuthology.orchestra.run.smithi114.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-15T21:14:25.024 INFO:teuthology.orchestra.run.smithi114.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-15T21:14:25.027 INFO:teuthology.orchestra.run.smithi114.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-15T21:14:25.028 INFO:teuthology.orchestra.run.smithi114.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-15T21:14:25.028 INFO:teuthology.orchestra.run.smithi114.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-15T21:14:25.037 INFO:teuthology.orchestra.run.smithi114.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-15T21:14:25.037 INFO:teuthology.orchestra.run.smithi114.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-15T21:14:25.040 INFO:teuthology.orchestra.run.smithi114.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-15T21:14:25.041 INFO:teuthology.orchestra.run.smithi114.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4930-gfc3e0bff-1jammy [112 kB] 2024-09-15T21:14:25.043 INFO:teuthology.orchestra.run.smithi114.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4930-gfc3e0bff-1jammy [473 kB] 2024-09-15T21:14:25.049 INFO:teuthology.orchestra.run.smithi114.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4930-gfc3e0bff-1jammy [28.2 MB] 2024-09-15T21:14:25.054 INFO:teuthology.orchestra.run.smithi114.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T21:14:25.054 INFO:teuthology.orchestra.run.smithi114.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-15T21:14:25.055 INFO:teuthology.orchestra.run.smithi114.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-15T21:14:25.056 INFO:teuthology.orchestra.run.smithi114.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-15T21:14:25.057 INFO:teuthology.orchestra.run.smithi114.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-15T21:14:25.063 INFO:teuthology.orchestra.run.smithi114.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-15T21:14:25.063 INFO:teuthology.orchestra.run.smithi114.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-15T21:14:25.071 INFO:teuthology.orchestra.run.smithi114.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-15T21:14:25.073 INFO:teuthology.orchestra.run.smithi114.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-15T21:14:25.073 INFO:teuthology.orchestra.run.smithi114.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-15T21:14:25.735 INFO:teuthology.orchestra.run.smithi114.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4930-gfc3e0bff-1jammy [5,206 kB] 2024-09-15T21:14:25.809 INFO:teuthology.orchestra.run.smithi114.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4930-gfc3e0bff-1jammy [251 kB] 2024-09-15T21:14:25.811 INFO:teuthology.orchestra.run.smithi114.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [124 kB] 2024-09-15T21:14:25.813 INFO:teuthology.orchestra.run.smithi114.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4930-gfc3e0bff-1jammy [1,561 kB] 2024-09-15T21:14:25.826 INFO:teuthology.orchestra.run.smithi114.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4930-gfc3e0bff-1jammy [6,309 kB] 2024-09-15T21:14:25.945 INFO:teuthology.orchestra.run.smithi114.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4930-gfc3e0bff-1jammy [23.1 MB] 2024-09-15T21:14:26.437 INFO:teuthology.orchestra.run.smithi114.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4930-gfc3e0bff-1jammy [14.1 kB] 2024-09-15T21:14:26.438 INFO:teuthology.orchestra.run.smithi114.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [880 kB] 2024-09-15T21:14:26.450 INFO:teuthology.orchestra.run.smithi114.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4930-gfc3e0bff-1jammy [2,452 kB] 2024-09-15T21:14:26.485 INFO:teuthology.orchestra.run.smithi114.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4930-gfc3e0bff-1jammy [813 kB] 2024-09-15T21:14:26.494 INFO:teuthology.orchestra.run.smithi114.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4930-gfc3e0bff-1jammy [160 kB] 2024-09-15T21:14:26.497 INFO:teuthology.orchestra.run.smithi114.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4930-gfc3e0bff-1jammy [61.4 MB] 2024-09-15T21:14:27.850 INFO:teuthology.orchestra.run.smithi114.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4930-gfc3e0bff-1jammy [8,625 kB] 2024-09-15T21:14:27.959 INFO:teuthology.orchestra.run.smithi114.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4930-gfc3e0bff-1jammy [14.2 kB] 2024-09-15T21:14:27.959 INFO:teuthology.orchestra.run.smithi114.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4930-gfc3e0bff-1jammy [48.5 MB] 2024-09-15T21:14:29.007 INFO:teuthology.orchestra.run.smithi114.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4930-gfc3e0bff-1jammy [130 kB] 2024-09-15T21:14:29.009 INFO:teuthology.orchestra.run.smithi114.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4930-gfc3e0bff-1jammy [39.0 kB] 2024-09-15T21:14:29.011 INFO:teuthology.orchestra.run.smithi114.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4930-gfc3e0bff-1jammy [15.8 MB] 2024-09-15T21:14:29.310 INFO:teuthology.orchestra.run.smithi114.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4930-gfc3e0bff-1jammy [91.2 kB] 2024-09-15T21:14:29.887 INFO:teuthology.orchestra.run.smithi114.stdout:Fetched 244 MB in 5s (50.9 MB/s) 2024-09-15T21:14:30.061 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-15T21:14:30.161 INFO:teuthology.orchestra.run.smithi114.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-15T21:14:30.166 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-15T21:14:30.251 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:14:30.537 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T21:14:30.563 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-15T21:14:30.595 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:14:30.923 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T21:14:30.950 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-15T21:14:30.990 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:14:31.427 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T21:14:31.453 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:14:31.511 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:14:31.863 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T21:14:31.890 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:14:31.922 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:14:32.233 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T21:14:32.259 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-15T21:14:32.299 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:14:32.644 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-15T21:14:32.670 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-15T21:14:32.711 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:14:33.005 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librados2. 2024-09-15T21:14:33.020 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../007-librados2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:33.055 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:33.508 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libnbd0. 2024-09-15T21:14:33.534 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-15T21:14:33.574 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-15T21:14:33.877 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librbd1. 2024-09-15T21:14:33.904 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../009-librbd1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:33.944 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:34.380 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T21:14:34.406 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:34.446 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:34.690 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rados. 2024-09-15T21:14:34.706 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:34.740 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:35.060 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T21:14:35.086 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:14:35.127 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:35.395 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T21:14:35.411 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:35.454 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:35.748 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T21:14:35.774 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:14:35.815 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:36.110 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T21:14:36.136 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-15T21:14:36.176 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:14:36.462 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T21:14:36.489 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-15T21:14:36.545 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-15T21:14:36.806 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T21:14:36.822 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:36.864 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:37.209 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T21:14:37.236 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-15T21:14:37.284 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:14:37.545 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-15T21:14:37.572 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-15T21:14:37.603 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:14:38.131 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-15T21:14:38.158 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-15T21:14:38.198 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:14:38.526 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua5.1. 2024-09-15T21:14:38.552 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-15T21:14:38.592 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:14:38.904 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-any. 2024-09-15T21:14:38.919 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-15T21:14:38.954 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-15T21:14:39.181 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package zip. 2024-09-15T21:14:39.197 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-15T21:14:39.239 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking zip (3.0-12build2) ... 2024-09-15T21:14:39.634 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package unzip. 2024-09-15T21:14:39.661 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-15T21:14:39.701 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:14:40.088 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package luarocks. 2024-09-15T21:14:40.114 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-15T21:14:40.154 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:14:40.512 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package librgw2. 2024-09-15T21:14:40.528 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../026-librgw2_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:40.562 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:41.106 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T21:14:41.121 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:41.164 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:41.492 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-15T21:14:41.519 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-15T21:14:41.558 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:14:41.845 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T21:14:41.873 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:41.911 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:42.331 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-common. 2024-09-15T21:14:42.358 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:42.398 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:43.879 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-base. 2024-09-15T21:14:43.906 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:43.954 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:44.507 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-15T21:14:44.533 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-15T21:14:44.574 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:14:44.835 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cheroot. 2024-09-15T21:14:44.861 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-15T21:14:44.893 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:14:45.188 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-15T21:14:45.214 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-15T21:14:45.254 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:14:45.574 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-15T21:14:45.600 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-15T21:14:45.640 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:14:45.943 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-15T21:14:45.969 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-15T21:14:46.018 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:14:46.279 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tempora. 2024-09-15T21:14:46.305 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-15T21:14:46.346 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-15T21:14:46.590 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-portend. 2024-09-15T21:14:46.616 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-15T21:14:46.657 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-15T21:14:46.976 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-15T21:14:47.002 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-15T21:14:47.043 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-15T21:14:47.314 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T21:14:47.370 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-15T21:14:47.373 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:14:47.709 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T21:14:47.735 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-15T21:14:47.776 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-15T21:14:48.096 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T21:14:48.122 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-15T21:14:48.164 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-15T21:14:48.582 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T21:14:48.608 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-15T21:14:48.640 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:14:48.926 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-mako. 2024-09-15T21:14:48.952 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-15T21:14:48.992 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:14:49.279 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T21:14:49.294 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T21:14:49.329 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:14:49.598 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T21:14:49.625 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-15T21:14:49.665 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:14:49.959 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webob. 2024-09-15T21:14:49.986 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-15T21:14:50.026 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:14:50.370 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T21:14:50.396 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-15T21:14:50.440 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:14:50.740 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T21:14:50.755 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-15T21:14:50.790 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:14:51.026 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-paste. 2024-09-15T21:14:51.053 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-15T21:14:51.084 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:14:51.387 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T21:14:51.402 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-15T21:14:51.454 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:14:51.740 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T21:14:51.766 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-15T21:14:51.806 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:14:52.109 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T21:14:52.136 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-15T21:14:52.176 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-15T21:14:52.471 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T21:14:52.497 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-15T21:14:52.537 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:14:52.892 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-certifi. 2024-09-15T21:14:52.919 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-15T21:14:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-15T21:14:53.253 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-idna. 2024-09-15T21:14:53.279 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-15T21:14:53.311 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:14:53.639 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-urllib3. 2024-09-15T21:14:53.666 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-15T21:14:53.705 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:14:54.025 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-requests. 2024-09-15T21:14:54.052 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-15T21:14:54.092 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:14:54.403 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T21:14:54.430 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-15T21:14:54.490 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:14:54.789 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T21:14:54.805 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:14:54.847 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:55.185 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T21:14:55.212 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:55.243 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:55.555 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T21:14:55.581 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-15T21:14:55.621 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:14:55.908 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T21:14:55.934 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-15T21:14:55.974 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-15T21:14:56.310 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T21:14:56.337 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:56.376 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:56.730 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T21:14:56.745 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:56.788 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:57.391 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T21:14:57.418 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:57.459 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:59.161 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph. 2024-09-15T21:14:59.177 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../067-ceph_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:59.220 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:59.489 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T21:14:59.515 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:59.547 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:14:59.917 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T21:14:59.943 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:14:59.983 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:00.394 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package cephadm. 2024-09-15T21:15:00.421 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../070-cephadm_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:00.460 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:00.747 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-15T21:15:00.774 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-15T21:15:00.814 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-15T21:15:01.108 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-15T21:15:01.123 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:01.166 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:02.204 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-babel. 2024-09-15T21:15:02.231 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-15T21:15:02.262 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:02.557 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T21:15:02.584 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-15T21:15:02.615 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:15:02.918 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T21:15:02.945 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:02.977 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:03.305 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T21:15:03.332 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T21:15:03.363 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T21:15:03.641 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-routes. 2024-09-15T21:15:03.668 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-15T21:15:03.707 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:15:04.060 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T21:15:04.087 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:04.127 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:06.840 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-15T21:15:06.857 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-15T21:15:06.898 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:15:07.443 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T21:15:07.471 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-15T21:15:07.502 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:15:07.863 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-15T21:15:07.891 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-15T21:15:07.930 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:15:08.232 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T21:15:08.261 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-15T21:15:08.376 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:15:09.002 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T21:15:09.018 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:09.057 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:10.163 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T21:15:10.190 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-15T21:15:10.229 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-15T21:15:10.532 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T21:15:10.560 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-15T21:15:10.599 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:15:10.935 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T21:15:10.963 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-15T21:15:11.002 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:15:11.313 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T21:15:11.341 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-15T21:15:11.388 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-15T21:15:11.725 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T21:15:11.753 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-15T21:15:11.792 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-15T21:15:12.087 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T21:15:12.114 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-15T21:15:12.144 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:15:12.464 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T21:15:12.492 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-15T21:15:12.531 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-15T21:15:12.851 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T21:15:12.879 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-15T21:15:12.917 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:15:13.539 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T21:15:13.567 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:13.606 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:13.959 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T21:15:13.986 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-15T21:15:14.025 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:15:14.362 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T21:15:14.389 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:14.428 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:15:14.681 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package jq. 2024-09-15T21:15:14.697 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-15T21:15:14.739 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-15T21:15:14.975 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package socat. 2024-09-15T21:15:14.991 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-15T21:15:15.025 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:15:15.278 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T21:15:15.306 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-15T21:15:15.336 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:15:15.690 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-test. 2024-09-15T21:15:15.706 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:15.740 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:17.953 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T21:15:18.025 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4930-gfc3e0bff-1jammy_all.deb ... 2024-09-15T21:15:18.025 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:18.300 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T21:15:18.326 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:18.356 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:18.634 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-15T21:15:18.662 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-15T21:15:18.701 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:15:19.022 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-15T21:15:19.050 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-15T21:15:19.196 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:15:19.567 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T21:15:19.595 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-15T21:15:19.634 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:15:20.064 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package pkg-config. 2024-09-15T21:15:20.092 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-15T21:15:20.130 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:15:20.392 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-15T21:15:20.419 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-15T21:15:20.458 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:15:20.878 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-15T21:15:20.894 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-15T21:15:20.936 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-15T21:15:21.238 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T21:15:21.266 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T21:15:21.305 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T21:15:21.634 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T21:15:21.661 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-15T21:15:21.700 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:15:21.969 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T21:15:21.997 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-15T21:15:22.036 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-15T21:15:22.339 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-py. 2024-09-15T21:15:22.367 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-15T21:15:22.397 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-15T21:15:22.650 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pygments. 2024-09-15T21:15:22.675 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-15T21:15:22.708 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:15:23.154 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T21:15:23.182 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-15T21:15:23.220 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:15:23.532 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-toml. 2024-09-15T21:15:23.560 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-15T21:15:23.598 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-15T21:15:23.901 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T21:15:23.929 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-15T21:15:23.967 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:15:24.296 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T21:15:24.324 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-15T21:15:24.362 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:15:24.965 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T21:15:24.981 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-15T21:15:25.023 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:15:25.677 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package radosgw. 2024-09-15T21:15:25.693 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../117-radosgw_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:25.727 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T21:15:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4930-gfc3e0bff-1jammy_amd64.deb ... 2024-09-15T21:15:26.679 INFO:teuthology.orchestra.run.smithi114.stdout:Unpacking rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:26.987 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-15T21:15:27.228 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-15T21:15:27.354 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-15T21:15:27.616 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-15T21:15:27.961 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-15T21:15:28.371 INFO:teuthology.orchestra.run.smithi114.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-15T21:15:28.439 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:28.964 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user cephadm....done 2024-09-15T21:15:29.145 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-15T21:15:29.423 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-15T21:15:29.673 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-15T21:15:29.808 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-15T21:15:30.084 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T21:15:30.344 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-15T21:15:30.444 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-15T21:15:30.746 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-15T21:15:31.097 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-15T21:15:31.349 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-15T21:15:31.625 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-15T21:15:31.885 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-15T21:15:32.111 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-15T21:15:32.370 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-15T21:15:32.613 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-argparse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:32.927 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-15T21:15:33.174 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-15T21:15:33.301 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-15T21:15:33.426 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-15T21:15:33.552 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-15T21:15:33.803 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-15T21:15:34.212 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-15T21:15:34.243 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-15T21:15:34.306 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-15T21:15:34.432 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-15T21:15:34.808 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:34.934 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-15T21:15:35.185 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-15T21:15:35.446 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-15T21:15:35.763 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up zip (3.0-12build2) ... 2024-09-15T21:15:35.889 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-15T21:15:36.574 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-15T21:15:36.834 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-15T21:15:36.959 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-15T21:15:37.085 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-15T21:15:37.386 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-15T21:15:37.762 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-15T21:15:38.022 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-15T21:15:38.391 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-15T21:15:38.683 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-15T21:15:39.009 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-15T21:15:39.244 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-15T21:15:39.370 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:39.663 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-15T21:15:39.980 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-15T21:15:41.262 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-15T21:15:41.513 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:41.648 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-15T21:15:41.924 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-15T21:15:42.058 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-15T21:15:42.167 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-15T21:15:42.444 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-15T21:15:42.678 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-15T21:15:42.921 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:43.047 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-15T21:15:43.340 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-15T21:15:43.591 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-15T21:15:43.867 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-15T21:15:44.110 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T21:15:44.352 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-15T21:15:44.662 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-15T21:15:44.788 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-15T21:15:45.072 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-15T21:15:45.198 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-15T21:15:45.483 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-15T21:15:45.792 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-15T21:15:46.035 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-15T21:15:46.161 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-15T21:15:46.279 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-15T21:15:46.352 INFO:teuthology.orchestra.run.smithi114.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-15T21:15:46.597 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-15T21:15:46.723 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-15T21:15:47.116 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-15T21:15:47.392 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-15T21:15:47.493 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-15T21:15:47.728 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-15T21:15:47.854 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-15T21:15:48.163 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-15T21:15:48.523 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-15T21:15:48.798 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-15T21:15:48.933 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-15T21:15:49.201 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-15T21:15:49.546 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-15T21:15:49.805 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-15T21:15:50.207 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T21:15:50.492 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-15T21:15:50.811 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-15T21:15:50.920 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librados2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:51.030 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:51.156 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-15T21:15:51.492 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:51.593 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libradosstriper1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:51.694 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-15T21:15:51.963 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librbd1 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:52.122 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-modules-core (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:52.248 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:52.463 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:15:52.463 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T21:15:52.933 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up libcephfs-dev (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rados (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:53.184 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up librgw2 (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:53.310 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-15T21:15:54.447 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rbd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.573 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up rbd-fuse (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.699 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-rgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.825 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up python3-cephfs (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:54.926 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-common (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:Adding group ceph....done 2024-09-15T21:15:55.763 INFO:teuthology.orchestra.run.smithi114.stdout:Adding system user ceph....done 2024-09-15T21:15:55.942 INFO:teuthology.orchestra.run.smithi114.stdout:Setting system user ceph properties....done 2024-09-15T21:15:55.963 INFO:teuthology.orchestra.run.smithi114.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T21:15:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T21:15:56.445 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T21:15:56.877 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-test (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:56.987 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up radosgw (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:57.444 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:15:57.444 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:15:58.198 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-base (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:58.546 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T21:15:59.017 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mds (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.232 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:15:59.232 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T21:15:59.685 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:15:59.911 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:15:59.911 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T21:16:00.387 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-osd (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:00.655 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:00.655 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T21:16:01.296 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:01.393 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:01.528 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mon (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:01.741 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:01.741 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T21:16:02.237 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-cephadm (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:02.388 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:02.514 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-mgr-dashboard (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:02.665 INFO:teuthology.orchestra.run.smithi114.stdout:Setting up ceph-volume (19.3.0-4930-gfc3e0bff-1jammy) ... 2024-09-15T21:16:03.035 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-15T21:16:03.144 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-15T21:16:03.288 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-15T21:16:04.331 INFO:teuthology.orchestra.run.smithi114.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-15T21:16:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T21:16:05.008 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:05.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:16:05.154 DEBUG:teuthology.orchestra.run.smithi114:> dpkg-query -W -f '${Version}' ceph 2024-09-15T21:16:05.173 INFO:teuthology.orchestra.run.smithi114.stdout:19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:05.173 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930-gfc3e0bff-1jammy 2024-09-15T21:16:05.174 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930-gfc3e0bff-1jammy is installed. 2024-09-15T21:16:05.175 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:16:05.176 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:05.176 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:16:05.230 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:16:05.231 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:05.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:16:05.285 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:16:05.340 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:16:05.340 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:05.340 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:16:05.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:16:05.449 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:16:05.449 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:05.449 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:16:05.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:16:05.561 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:16:05.651 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:16:05.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-09-15T21:16:05.665 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:16:05.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:16:05.721 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:16:05.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:16:05.785 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:16:05.785 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:16:05.785 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674', 'branch': 'main', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zlib} rados supported-random-distro$/{ubuntu_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7905674', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 2855, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'sleep_before_teardown': 0, 'subset': '17936/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7498493279982dfe87f00616198a5967475f6169', 'targets': {'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNBz+BJB5K9tmFCvufx3TPBjyTqn4Umkn+kchncuxTpuiClZUxA6kHujRDfrJNZ0BrvOP3Rabzvoi3HAn8qjjI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_20:00:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:16:05.785 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:16:05.825 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:16:05.882 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:05.882 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:16:05.931 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:16:05.931 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-15T21:16:05.979 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:16:05.979 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-15 21:10:42.468412194 +0000 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-15 21:10:42.204414553 +0000 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-15 21:10:42.204414553 +0000 2024-09-15T21:16:05.980 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-15T21:16:05.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:16:06.037 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-15T21:16:06.037 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-15T21:16:06.037 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000442511 s, 1.2 MB/s 2024-09-15T21:16:06.038 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:16:06.089 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-15T21:16:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:16:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:06.138 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-15T21:16:06.138 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:06.138 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-15 21:10:42.712410016 +0000 2024-09-15T21:16:06.139 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-15 21:10:42.708410052 +0000 2024-09-15T21:16:06.139 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-15 21:10:42.708410052 +0000 2024-09-15T21:16:06.139 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-15T21:16:06.139 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:16:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-15T21:16:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-15T21:16:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000451074 s, 1.1 MB/s 2024-09-15T21:16:06.194 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:16:06.243 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-15T21:16:06.289 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-15 21:10:43.424403656 +0000 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-15 21:10:43.204405619 +0000 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-15 21:10:43.204405619 +0000 2024-09-15T21:16:06.290 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-15T21:16:06.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:16:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-15T21:16:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-15T21:16:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000442989 s, 1.2 MB/s 2024-09-15T21:16:06.345 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:16:06.395 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-15T21:16:06.441 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:16:06.441 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-15 21:10:43.880399581 +0000 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-15 21:10:43.644401690 +0000 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-15 21:10:43.644401690 +0000 2024-09-15T21:16:06.442 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: - 2024-09-15T21:16:06.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:16:06.496 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-15T21:16:06.496 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-15T21:16:06.496 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00038973 s, 1.3 MB/s 2024-09-15T21:16:06.497 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:16:06.547 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:06.547 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:16:06.547 INFO:tasks.ceph:Generating config... 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] mon client hunt interval max multiple = 2 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] mon mgr beacon grace = 90 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] ms inject socket failures = 1000 2024-09-15T21:16:06.549 INFO:tasks.ceph:[global] ms type = async 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mgr] debug monc = 10 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:16:06.550 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-15T21:16:06.550 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:16:06.551 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T21:16:06.552 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T21:16:06.552 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:16:06.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:16:06.620 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:16:06.630 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:16:06.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:16:06.761 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.114'), ('mon.b', '[v2:172.21.15.114:3301,v1:172.21.15.114:6790]'), ('mon.c', '[v2:172.21.15.114:3302,v1:172.21.15.114:6791]')] 2024-09-15T21:16:06.762 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790],[v2:172.21.15.114:3302,v1:172.21.15.114:6791]', 'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-15T21:16:06.764 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:06.764 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:16:06.810 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.114 --addv b '[v2:172.21.15.114:3301,v1:172.21.15.114:6790]' --addv c '[v2:172.21.15.114:3302,v1:172.21.15.114:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: generated fsid 84ba6ec4-065b-47f6-87f1-d15da93be3d3 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = quincy 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 0 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 84ba6ec4-065b-47f6-87f1-d15da93be3d3 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:last_changed 2024-09-15T21:16:06.895038+0000 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-15T21:16:06.895038+0000 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:min_mon_release 17 (quincy) 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:election_strategy: 3 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-15T21:16:06.896 INFO:teuthology.orchestra.run.smithi114.stdout:1: [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] mon.b 2024-09-15T21:16:06.897 INFO:teuthology.orchestra.run.smithi114.stdout:2: [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] mon.c 2024-09-15T21:16:06.897 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:16:06.898 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:16:06.946 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 84ba6ec4-065b-47f6-87f1-d15da93be3d3... 2024-09-15T21:16:06.947 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:16:07.019 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-15T21:16:07.019 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-09-15T21:16:07.019 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:16:07.019 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:16:07.020 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:16:07.021 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-09-15T21:16:07.022 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.114,[v2:172.21.15.114:3301,v1:172.21.15.114:6790],[v2:172.21.15.114:3302,v1:172.21.15.114:6791]" 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: mon client hunt interval max multiple = 2 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: mon election default strategy = 3 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr beacon grace = 90 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: ms bind msgr1 = False 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: ms bind msgr2 = True 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 1000 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: ms type = async 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 84ba6ec4-065b-47f6-87f1-d15da93be3d3 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-09-15T21:16:07.023 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.024 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.025 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression algorithm = zlib 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore compression mode = aggressive 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore write v2 random = True 2024-09-15T21:16:07.026 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore zero block detection = True 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify cached snaps = True 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify missing on start = True 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock override recovery settings = True 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd op queue = debug_random 2024-09-15T21:16:07.027 INFO:teuthology.orchestra.run.smithi114.stdout: osd op queue cut off = debug_random 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug monc = 10 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-09-15T21:16:07.028 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:16:07.029 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.030 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: mon scrub interval = 300 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-09-15T21:16:07.031 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-09-15T21:16:07.032 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:16:07.033 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:16:07.127 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:16:07.127 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:07.128 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:16:07.133 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:07.133 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:16:07.182 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:16:07.183 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:07.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:16:07.183 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:16:07.245 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:07.246 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:07.294 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:16:07.294 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:16:07.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:16:07.377 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:16:07.380 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:16:07.380 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:16:07.380 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:16:07.380 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:16:07.381 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:07.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:07.433 INFO:tasks.ceph:role: osd.0 2024-09-15T21:16:07.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:16:07.434 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:07.498 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:07.972 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-15T21:16:07.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-15T21:16:07.974 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:08.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:08.146 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:08.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:08.147 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:08.202 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:08.202 INFO:tasks.ceph:role: osd.1 2024-09-15T21:16:08.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:16:08.202 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:16:08.268 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:08.269 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:08.709 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-15T21:16:08.711 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-15T21:16:08.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:08.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:08.853 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:08.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:08.855 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:08.907 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:16:08.907 INFO:tasks.ceph:role: osd.2 2024-09-15T21:16:08.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:16:08.908 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:16:08.969 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:16:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:16:08.970 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:16:09.432 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-15T21:16:09.434 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-15T21:16:09.434 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:09.550 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:09.560 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T21:16:09.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:16:09.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:09.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:09.671+0000 7f6ab89777c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:16:09.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:09.675+0000 7f6ab89777c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:16:09.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:09.675+0000 7f6ab89777c0 -1 bdev(0x556edb7bd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:09.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:09.675+0000 7f6ab89777c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:16:12.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:16:12.844 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:12.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:12.923+0000 7fb600f967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:16:12.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:12.923+0000 7fb600f967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:16:12.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:12.923+0000 7fb600f967c0 -1 bdev(0x556a1ed89800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:12.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:12.927+0000 7fb600f967c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:16:16.098 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:16:16.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:16.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:16.239+0000 7f1261f037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:16:16.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:16.239+0000 7f1261f037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:16:16.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:16.239+0000 7f1261f037c0 -1 bdev(0x562b3a6c9800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T21:16:16.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:16:16.239+0000 7f1261f037c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:16:19.396 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:16:19.449 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:16:19.450 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:19.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:16:19.505 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:19.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:16:19.561 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:19.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:16:19.621 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:19.621 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:16:19.676 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:16:19.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:16:19.732 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-09-15T21:16:19.732 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCXTudmLV52FhAAGmYbYrOYIIO9U9EhICLErA== 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCZTudmJnA7KBAACNEZ5VWukapxEOAFuBXjUw== 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCcTudmLM5INxAAACiYZjUymz6SVVLWL7jMwA== 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-09-15T21:16:19.733 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCgTudmovFgDhAA9qkX3yGfYzmwAJImpmcTmw== 2024-09-15T21:16:19.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:16:19.822 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:16:19.863 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:16:19.947 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:16:20.033 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:16:20.033 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:16:20.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:16:20.187 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:16:20.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:16:20.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:16:20.339 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:16:20.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:16:20.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:16:20.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:16:20.501 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:16:20.549 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:16:20.549 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:16:20.549 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:16:20.594 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:16:20.594 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:16:20.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:16:20.596 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:16:20.596 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:16:20.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:16:20.598 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:16:20.598 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph config set mgr mgr_pool false --force 2024-09-15T21:16:27.444 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:16:27.444 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:16:27.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:16:27.447 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:16:27.447 DEBUG:tasks.ceph:set 0 configs 2024-09-15T21:16:27.447 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:16:27.447 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:16:27.638 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-09-15T21:16:27.655 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:16:27.656 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:16:27.656 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:27.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:16:27.666 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:27.666 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:16:27.723 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-15T21:16:27.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:16:27.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 3944bfc1-02b1-40e5-9748-a6b5781fcfd3 0 2024-09-15T21:16:28.019 INFO:teuthology.orchestra.run.smithi114.stdout:0 2024-09-15T21:16:28.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 8e707e07-76a7-4d0b-b923-42e77df0f4eb 1 2024-09-15T21:16:28.236 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-15T21:16:28.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 7647c42f-573c-4e9b-a6e4-b65509427795 2 2024-09-15T21:16:28.450 INFO:teuthology.orchestra.run.smithi114.stdout:2 2024-09-15T21:16:28.467 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:16:28.468 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:16:28.469 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:16:28.470 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:16:28.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:16:28.471 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:16:28.472 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:16:28.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:16:28.474 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:16:28.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:28.654 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:28.654 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:28.447108+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:28.670 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:16:28.670 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:16:28.972 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:29.146 INFO:teuthology.misc.health.smithi114.stdout: 2024-09-15T21:16:29.146 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":5,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:28.447108+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:29.161 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-15T21:16:29.882 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:16:29.878+0000 7f3b8416f7c0 -1 Falling back to public interface 2024-09-15T21:16:30.070 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:16:30.066+0000 7f2c7fed97c0 -1 Falling back to public interface 2024-09-15T21:16:30.102 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:16:30.098+0000 7f04c65d17c0 -1 Falling back to public interface 2024-09-15T21:16:30.384 INFO:tasks.ceph.mgr.x.smithi114.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-15T21:16:30.384 INFO:tasks.ceph.mgr.x.smithi114.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-15T21:16:30.384 INFO:tasks.ceph.mgr.x.smithi114.stderr: from numpy import show_config as show_numpy_config 2024-09-15T21:16:33.284 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-15T21:16:33.278+0000 7fa0fe050640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:16:33.290 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-15T21:16:33.286+0000 7fa0fe050640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:16:33.909 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:16:33.906+0000 7f3b8416f7c0 -1 osd.1 0 log_to_monitors true 2024-09-15T21:16:34.092 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:16:34.086+0000 7f04c65d17c0 -1 osd.0 0 log_to_monitors true 2024-09-15T21:16:34.539 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:16:34.534+0000 7f2c7fed97c0 -1 osd.2 0 log_to_monitors true 2024-09-15T21:16:35.344 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:16:35.338+0000 7f3b80118640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:16:35.355 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:16:35.350+0000 7f3b7bf52640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:16:35.463 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:35.784 INFO:teuthology.misc.health.smithi114.stdout: 2024-09-15T21:16:35.785 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":7,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:35.329026+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:35.802 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-15T21:16:36.336 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:16:36.330+0000 7f3b77a2e640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:16:38.046 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:16:38.042+0000 7f04c2d8c640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:16:38.050 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:16:38.046+0000 7f04ba3ac640 -1 osd.0 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:16:39.014 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:16:39.010+0000 7f2c7be82640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:16:39.018 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:16:39.014+0000 7f2c73cb4640 -1 osd.2 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:16:42.105 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:42.414 INFO:teuthology.misc.health.smithi114.stdout: 2024-09-15T21:16:42.415 INFO:teuthology.misc.health.smithi114.stdout:{"epoch":11,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:40.058040+0000","last_up_change":"2024-09-15T21:16:39.048022+0000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3116989521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":3116989521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":3116989521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3116989521}]},"public_addr":"172.21.15.114:6808/3116989521","cluster_addr":"172.21.15.114:6809/3116989521","heartbeat_back_addr":"172.21.15.114:6811/3116989521","heartbeat_front_addr":"172.21.15.114:6810/3116989521","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2337399995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":2337399995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2337399995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2337399995}]},"public_addr":"172.21.15.114:6804/2337399995","cluster_addr":"172.21.15.114:6805/2337399995","heartbeat_back_addr":"172.21.15.114:6807/2337399995","heartbeat_front_addr":"172.21.15.114:6806/2337399995","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:42.432 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-15T21:16:42.432 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:16:42.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:16:43.345 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'rbd' created 2024-09-15T21:16:43.367 DEBUG:teuthology.orchestra.run.smithi114:> rbd --cluster ceph pool init rbd 2024-09-15T21:16:46.371 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:16:46.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T21:16:46.372 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:16:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:46.707 INFO:teuthology.orchestra.run.smithi114.stdout:[{"version":4,"timestamp":"2024-09-15T21:16:39.014245+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"8446.970070"}]}] 2024-09-15T21:16:46.707 INFO:tasks.ceph_manager:config epoch is 4 2024-09-15T21:16:46.708 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:16:46.708 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:16:46.708 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:16:47.070 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:47.087 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":4,"flags":0,"active_gid":4110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":535665302}]},"active_addr":"172.21.15.114:6812/535665302","active_change":"2024-09-15T21:16:32.261982+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2621397071}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":4203325137}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":170354141}]}]} 2024-09-15T21:16:47.091 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:16:47.092 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:16:47.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:47.400 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:46.339959+0000","last_up_change":"2024-09-15T21:16:39.048022+0000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3116989521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":3116989521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":3116989521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3116989521}]},"public_addr":"172.21.15.114:6808/3116989521","cluster_addr":"172.21.15.114:6809/3116989521","heartbeat_back_addr":"172.21.15.114:6811/3116989521","heartbeat_front_addr":"172.21.15.114:6810/3116989521","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2337399995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":2337399995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2337399995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2337399995}]},"public_addr":"172.21.15.114:6804/2337399995","cluster_addr":"172.21.15.114:6805/2337399995","heartbeat_back_addr":"172.21.15.114:6807/2337399995","heartbeat_front_addr":"172.21.15.114:6806/2337399995","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:47.419 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:16:47.419 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:47.715 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:47.715 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":15,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:46.339959+0000","last_up_change":"2024-09-15T21:16:39.048022+0000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3116989521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":3116989521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":3116989521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3116989521}]},"public_addr":"172.21.15.114:6808/3116989521","cluster_addr":"172.21.15.114:6809/3116989521","heartbeat_back_addr":"172.21.15.114:6811/3116989521","heartbeat_front_addr":"172.21.15.114:6810/3116989521","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2337399995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":2337399995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2337399995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2337399995}]},"public_addr":"172.21.15.114:6804/2337399995","cluster_addr":"172.21.15.114:6805/2337399995","heartbeat_back_addr":"172.21.15.114:6807/2337399995","heartbeat_front_addr":"172.21.15.114:6806/2337399995","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:47.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:16:47.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:16:47.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:16:47.888 INFO:teuthology.orchestra.run.smithi114.stdout:42949672965 2024-09-15T21:16:47.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:16:47.896 INFO:teuthology.orchestra.run.smithi114.stdout:34359738373 2024-09-15T21:16:47.896 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:16:47.898 INFO:teuthology.orchestra.run.smithi114.stdout:42949672965 2024-09-15T21:16:47.899 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:16:48.184 INFO:teuthology.orchestra.run.smithi114.stdout:42949672964 2024-09-15T21:16:48.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672964 for osd.2 2024-09-15T21:16:48.225 INFO:teuthology.orchestra.run.smithi114.stdout:42949672964 2024-09-15T21:16:48.225 INFO:teuthology.orchestra.run.smithi114.stdout:34359738371 2024-09-15T21:16:48.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672964 for osd.0 2024-09-15T21:16:48.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.1 2024-09-15T21:16:49.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:16:49.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:16:49.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:16:49.518 INFO:teuthology.orchestra.run.smithi114.stdout:42949672965 2024-09-15T21:16:49.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.2 2024-09-15T21:16:49.536 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:49.544 INFO:teuthology.orchestra.run.smithi114.stdout:34359738373 2024-09-15T21:16:49.555 INFO:teuthology.orchestra.run.smithi114.stdout:42949672965 2024-09-15T21:16:49.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2024-09-15T21:16:49.562 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:49.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.0 2024-09-15T21:16:49.573 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:49.573 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:16:49.574 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:16:49.891 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:16:49.910 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T21:16:48.273551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81488,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034032,"statfs":{"total":289910292480,"available":289826848768,"internally_reserved":0,"allocated":401408,"data_stored":92432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.355617+0000","last_change":"2024-09-15T21:16:46.355617+0000","last_active":"2024-09-15T21:16:46.355617+0000","last_peered":"2024-09-15T21:16:46.355617+0000","last_clean":"2024-09-15T21:16:46.355617+0000","last_became_active":"2024-09-15T21:16:44.351500+0000","last_became_peered":"2024-09-15T21:16:44.351500+0000","last_unstale":"2024-09-15T21:16:46.355617+0000","last_undegraded":"2024-09-15T21:16:46.355617+0000","last_fullsized":"2024-09-15T21:16:46.355617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:20.646020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.355661+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:46.355661+0000","last_peered":"2024-09-15T21:16:46.355661+0000","last_clean":"2024-09-15T21:16:46.355661+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:46.355661+0000","last_undegraded":"2024-09-15T21:16:46.355661+0000","last_fullsized":"2024-09-15T21:16:46.355661+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354467+0000","last_change":"2024-09-15T21:16:46.355239+0000","last_active":"2024-09-15T21:16:46.354467+0000","last_peered":"2024-09-15T21:16:46.354467+0000","last_clean":"2024-09-15T21:16:46.354467+0000","last_became_active":"2024-09-15T21:16:44.348704+0000","last_became_peered":"2024-09-15T21:16:44.348704+0000","last_unstale":"2024-09-15T21:16:46.354467+0000","last_undegraded":"2024-09-15T21:16:46.354467+0000","last_fullsized":"2024-09-15T21:16:46.354467+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:17.678766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028794379999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.356388+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:46.356388+0000","last_peered":"2024-09-15T21:16:46.356388+0000","last_clean":"2024-09-15T21:16:46.356388+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:46.356388+0000","last_undegraded":"2024-09-15T21:16:46.356388+0000","last_fullsized":"2024-09-15T21:16:46.356388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.353967+0000","last_change":"2024-09-15T21:16:46.353967+0000","last_active":"2024-09-15T21:16:46.353967+0000","last_peered":"2024-09-15T21:16:46.353967+0000","last_clean":"2024-09-15T21:16:46.353967+0000","last_became_active":"2024-09-15T21:16:44.356586+0000","last_became_peered":"2024-09-15T21:16:44.356586+0000","last_unstale":"2024-09-15T21:16:46.353967+0000","last_undegraded":"2024-09-15T21:16:46.353967+0000","last_fullsized":"2024-09-15T21:16:46.353967+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.614015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.353429+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:46.353429+0000","last_peered":"2024-09-15T21:16:46.353429+0000","last_clean":"2024-09-15T21:16:46.353429+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:46.353429+0000","last_undegraded":"2024-09-15T21:16:46.353429+0000","last_fullsized":"2024-09-15T21:16:46.353429+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354319+0000","last_change":"2024-09-15T21:16:46.355174+0000","last_active":"2024-09-15T21:16:46.354319+0000","last_peered":"2024-09-15T21:16:46.354319+0000","last_clean":"2024-09-15T21:16:46.354319+0000","last_became_active":"2024-09-15T21:16:44.348325+0000","last_became_peered":"2024-09-15T21:16:44.348325+0000","last_unstale":"2024-09-15T21:16:46.354319+0000","last_undegraded":"2024-09-15T21:16:46.354319+0000","last_fullsized":"2024-09-15T21:16:46.354319+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:08.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037593539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354325+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:46.354325+0000","last_peered":"2024-09-15T21:16:46.354325+0000","last_clean":"2024-09-15T21:16:46.354325+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:46.354325+0000","last_undegraded":"2024-09-15T21:16:46.354325+0000","last_fullsized":"2024-09-15T21:16:46.354325+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":131072,"data_stored":30798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672965,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":135168,"data_stored":30817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":30817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:16:49.911 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:16:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:50.185 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:16:50.201 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T21:16:48.273551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81488,"kb_used_data":392,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034032,"statfs":{"total":289910292480,"available":289826848768,"internally_reserved":0,"allocated":401408,"data_stored":92432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.355617+0000","last_change":"2024-09-15T21:16:46.355617+0000","last_active":"2024-09-15T21:16:46.355617+0000","last_peered":"2024-09-15T21:16:46.355617+0000","last_clean":"2024-09-15T21:16:46.355617+0000","last_became_active":"2024-09-15T21:16:44.351500+0000","last_became_peered":"2024-09-15T21:16:44.351500+0000","last_unstale":"2024-09-15T21:16:46.355617+0000","last_undegraded":"2024-09-15T21:16:46.355617+0000","last_fullsized":"2024-09-15T21:16:46.355617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:20.646020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.355661+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:46.355661+0000","last_peered":"2024-09-15T21:16:46.355661+0000","last_clean":"2024-09-15T21:16:46.355661+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:46.355661+0000","last_undegraded":"2024-09-15T21:16:46.355661+0000","last_fullsized":"2024-09-15T21:16:46.355661+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354467+0000","last_change":"2024-09-15T21:16:46.355239+0000","last_active":"2024-09-15T21:16:46.354467+0000","last_peered":"2024-09-15T21:16:46.354467+0000","last_clean":"2024-09-15T21:16:46.354467+0000","last_became_active":"2024-09-15T21:16:44.348704+0000","last_became_peered":"2024-09-15T21:16:44.348704+0000","last_unstale":"2024-09-15T21:16:46.354467+0000","last_undegraded":"2024-09-15T21:16:46.354467+0000","last_fullsized":"2024-09-15T21:16:46.354467+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:17.678766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028794379999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.356388+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:46.356388+0000","last_peered":"2024-09-15T21:16:46.356388+0000","last_clean":"2024-09-15T21:16:46.356388+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:46.356388+0000","last_undegraded":"2024-09-15T21:16:46.356388+0000","last_fullsized":"2024-09-15T21:16:46.356388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.353967+0000","last_change":"2024-09-15T21:16:46.353967+0000","last_active":"2024-09-15T21:16:46.353967+0000","last_peered":"2024-09-15T21:16:46.353967+0000","last_clean":"2024-09-15T21:16:46.353967+0000","last_became_active":"2024-09-15T21:16:44.356586+0000","last_became_peered":"2024-09-15T21:16:44.356586+0000","last_unstale":"2024-09-15T21:16:46.353967+0000","last_undegraded":"2024-09-15T21:16:46.353967+0000","last_fullsized":"2024-09-15T21:16:46.353967+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.614015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.353429+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:46.353429+0000","last_peered":"2024-09-15T21:16:46.353429+0000","last_clean":"2024-09-15T21:16:46.353429+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:46.353429+0000","last_undegraded":"2024-09-15T21:16:46.353429+0000","last_fullsized":"2024-09-15T21:16:46.353429+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354319+0000","last_change":"2024-09-15T21:16:46.355174+0000","last_active":"2024-09-15T21:16:46.354319+0000","last_peered":"2024-09-15T21:16:46.354319+0000","last_clean":"2024-09-15T21:16:46.354319+0000","last_became_active":"2024-09-15T21:16:44.348325+0000","last_became_peered":"2024-09-15T21:16:44.348325+0000","last_unstale":"2024-09-15T21:16:46.354319+0000","last_undegraded":"2024-09-15T21:16:46.354319+0000","last_fullsized":"2024-09-15T21:16:46.354319+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:08.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037593539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:16:46.354325+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:46.354325+0000","last_peered":"2024-09-15T21:16:46.354325+0000","last_clean":"2024-09-15T21:16:46.354325+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:46.354325+0000","last_undegraded":"2024-09-15T21:16:46.354325+0000","last_fullsized":"2024-09-15T21:16:46.354325+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672965,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":131072,"data_stored":30798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672965,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":135168,"data_stored":30817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":30817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:16:50.202 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:16:50.202 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:16:50.202 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:16:50.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:16:50.531 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:50.532 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:16:50.549 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:16:50.549 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-15T21:16:50.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:16:50.878 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:50.878 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:16:50.671666+0000","last_up_change":"2024-09-15T21:16:39.048022+0000","last_in_change":"2024-09-15T21:16:28.447108+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":3116989521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":3116989521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":3116989521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":3116989521}]},"public_addr":"172.21.15.114:6808/3116989521","cluster_addr":"172.21.15.114:6809/3116989521","heartbeat_back_addr":"172.21.15.114:6811/3116989521","heartbeat_front_addr":"172.21.15.114:6810/3116989521","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2337399995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":2337399995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2337399995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2337399995}]},"public_addr":"172.21.15.114:6804/2337399995","cluster_addr":"172.21.15.114:6805/2337399995","heartbeat_back_addr":"172.21.15.114:6807/2337399995","heartbeat_front_addr":"172.21.15.114:6806/2337399995","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:16:50.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:16:42.738751+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 15, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T21:16:50.898 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T21:16:51.178 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 8 2024-09-15T21:16:51.196 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 16 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 84ba6ec4-065b-47f6-87f1-d15da93be3d3 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-15T21:16:26.966560+0000 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-09-15T21:16:50.671666+0000 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:crush_version 5 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:full_ratio 0.95 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:backfillfull_ratio 0.9 2024-09-15T21:16:51.481 INFO:teuthology.orchestra.run.smithi114.stdout:nearfull_ratio 0.85 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:require_min_compat_client luminous 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:min_compat_client jewel 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:require_osd_release squid 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:stretch_mode_enabled false 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:max_osd 3 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6808/3116989521 v2:172.21.15.114:6809/3116989521 exists,up 3944bfc1-02b1-40e5-9748-a6b5781fcfd3 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 up in weight 1 up_from 8 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6800/2054907799 v2:172.21.15.114:6801/2054907799 exists,up 8e707e07-76a7-4d0b-b923-42e77df0f4eb 2024-09-15T21:16:51.482 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6804/2337399995 v2:172.21.15.114:6805/2337399995 exists,up 7647c42f-573c-4e9b-a6e4-b65509427795 2024-09-15T21:16:51.498 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6808/3116989521 v2:172.21.15.114:6809/3116989521 exists,up 3944bfc1-02b1-40e5-9748-a6b5781fcfd3', 'osd.1 up in weight 1 up_from 8 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6800/2054907799 v2:172.21.15.114:6801/2054907799 exists,up 8e707e07-76a7-4d0b-b923-42e77df0f4eb', 'osd.2 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.114:6804/2337399995 v2:172.21.15.114:6805/2337399995 exists,up 7647c42f-573c-4e9b-a6e4-b65509427795'] 2024-09-15T21:16:51.498 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:16:51.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:16:51.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:16:51.646 INFO:teuthology.orchestra.run.smithi114.stdout:42949672967 2024-09-15T21:16:51.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:16:51.675 INFO:teuthology.orchestra.run.smithi114.stdout:34359738375 2024-09-15T21:16:51.675 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:16:51.676 INFO:teuthology.orchestra.run.smithi114.stdout:42949672967 2024-09-15T21:16:51.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:16:51.971 INFO:teuthology.orchestra.run.smithi114.stdout:42949672966 2024-09-15T21:16:51.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672966 for osd.2 2024-09-15T21:16:52.004 INFO:teuthology.orchestra.run.smithi114.stdout:42949672966 2024-09-15T21:16:52.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672966 for osd.0 2024-09-15T21:16:52.032 INFO:teuthology.orchestra.run.smithi114.stdout:34359738373 2024-09-15T21:16:52.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738373 for osd.1 2024-09-15T21:16:52.990 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:16:53.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:16:53.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:16:53.302 INFO:teuthology.orchestra.run.smithi114.stdout:42949672967 2024-09-15T21:16:53.320 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672967 for osd.2 2024-09-15T21:16:53.320 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:53.347 INFO:teuthology.orchestra.run.smithi114.stdout:42949672967 2024-09-15T21:16:53.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672967 got 42949672967 for osd.0 2024-09-15T21:16:53.364 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:53.371 INFO:teuthology.orchestra.run.smithi114.stdout:34359738375 2024-09-15T21:16:53.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.1 2024-09-15T21:16:53.389 DEBUG:teuthology.parallel:result is None 2024-09-15T21:16:53.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-15T21:16:53.389 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:16:53.719 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:53.719 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:16:53.736 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-15T21:16:52.274406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81500,"kb_used_data":416,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034020,"statfs":{"total":289910292480,"available":289826836480,"internally_reserved":0,"allocated":425984,"data_stored":99602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000854"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.682850+0000","last_change":"2024-09-15T21:16:46.355617+0000","last_active":"2024-09-15T21:16:50.682850+0000","last_peered":"2024-09-15T21:16:50.682850+0000","last_clean":"2024-09-15T21:16:50.682850+0000","last_became_active":"2024-09-15T21:16:44.351500+0000","last_became_peered":"2024-09-15T21:16:44.351500+0000","last_unstale":"2024-09-15T21:16:50.682850+0000","last_undegraded":"2024-09-15T21:16:50.682850+0000","last_fullsized":"2024-09-15T21:16:50.682850+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:20.646020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681755+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:50.681755+0000","last_peered":"2024-09-15T21:16:50.681755+0000","last_clean":"2024-09-15T21:16:50.681755+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:50.681755+0000","last_undegraded":"2024-09-15T21:16:50.681755+0000","last_fullsized":"2024-09-15T21:16:50.681755+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.677941+0000","last_change":"2024-09-15T21:16:46.355239+0000","last_active":"2024-09-15T21:16:50.677941+0000","last_peered":"2024-09-15T21:16:50.677941+0000","last_clean":"2024-09-15T21:16:50.677941+0000","last_became_active":"2024-09-15T21:16:44.348704+0000","last_became_peered":"2024-09-15T21:16:44.348704+0000","last_unstale":"2024-09-15T21:16:50.677941+0000","last_undegraded":"2024-09-15T21:16:50.677941+0000","last_fullsized":"2024-09-15T21:16:50.677941+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:17.678766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028794379999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681237+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:50.681237+0000","last_peered":"2024-09-15T21:16:50.681237+0000","last_clean":"2024-09-15T21:16:50.681237+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:50.681237+0000","last_undegraded":"2024-09-15T21:16:50.681237+0000","last_fullsized":"2024-09-15T21:16:50.681237+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":23,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681198+0000","last_change":"2024-09-15T21:16:46.353967+0000","last_active":"2024-09-15T21:16:50.681198+0000","last_peered":"2024-09-15T21:16:50.681198+0000","last_clean":"2024-09-15T21:16:50.681198+0000","last_became_active":"2024-09-15T21:16:44.356586+0000","last_became_peered":"2024-09-15T21:16:44.356586+0000","last_unstale":"2024-09-15T21:16:50.681198+0000","last_undegraded":"2024-09-15T21:16:50.681198+0000","last_fullsized":"2024-09-15T21:16:50.681198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.614015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":24,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681049+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:50.681049+0000","last_peered":"2024-09-15T21:16:50.681049+0000","last_clean":"2024-09-15T21:16:50.681049+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:50.681049+0000","last_undegraded":"2024-09-15T21:16:50.681049+0000","last_fullsized":"2024-09-15T21:16:50.681049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.677942+0000","last_change":"2024-09-15T21:16:46.355174+0000","last_active":"2024-09-15T21:16:50.677942+0000","last_peered":"2024-09-15T21:16:50.677942+0000","last_clean":"2024-09-15T21:16:50.677942+0000","last_became_active":"2024-09-15T21:16:44.348325+0000","last_became_peered":"2024-09-15T21:16:44.348325+0000","last_unstale":"2024-09-15T21:16:50.677942+0000","last_undegraded":"2024-09-15T21:16:50.677942+0000","last_fullsized":"2024-09-15T21:16:50.677942+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:08.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037593539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.682834+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:50.682834+0000","last_peered":"2024-09-15T21:16:50.682834+0000","last_clean":"2024-09-15T21:16:50.682834+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:50.682834+0000","last_undegraded":"2024-09-15T21:16:50.682834+0000","last_fullsized":"2024-09-15T21:16:50.682834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":33188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":143360,"data_stored":33207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":143360,"data_stored":33207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:16:53.737 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:16:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:16:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:16:54.025 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-15T21:16:52.274406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81500,"kb_used_data":416,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034020,"statfs":{"total":289910292480,"available":289826836480,"internally_reserved":0,"allocated":425984,"data_stored":99602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000854"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.682850+0000","last_change":"2024-09-15T21:16:46.355617+0000","last_active":"2024-09-15T21:16:50.682850+0000","last_peered":"2024-09-15T21:16:50.682850+0000","last_clean":"2024-09-15T21:16:50.682850+0000","last_became_active":"2024-09-15T21:16:44.351500+0000","last_became_peered":"2024-09-15T21:16:44.351500+0000","last_unstale":"2024-09-15T21:16:50.682850+0000","last_undegraded":"2024-09-15T21:16:50.682850+0000","last_fullsized":"2024-09-15T21:16:50.682850+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:20.646020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681755+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:50.681755+0000","last_peered":"2024-09-15T21:16:50.681755+0000","last_clean":"2024-09-15T21:16:50.681755+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:50.681755+0000","last_undegraded":"2024-09-15T21:16:50.681755+0000","last_fullsized":"2024-09-15T21:16:50.681755+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.677941+0000","last_change":"2024-09-15T21:16:46.355239+0000","last_active":"2024-09-15T21:16:50.677941+0000","last_peered":"2024-09-15T21:16:50.677941+0000","last_clean":"2024-09-15T21:16:50.677941+0000","last_became_active":"2024-09-15T21:16:44.348704+0000","last_became_peered":"2024-09-15T21:16:44.348704+0000","last_unstale":"2024-09-15T21:16:50.677941+0000","last_undegraded":"2024-09-15T21:16:50.677941+0000","last_fullsized":"2024-09-15T21:16:50.677941+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:17.678766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028794379999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681237+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:50.681237+0000","last_peered":"2024-09-15T21:16:50.681237+0000","last_clean":"2024-09-15T21:16:50.681237+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:50.681237+0000","last_undegraded":"2024-09-15T21:16:50.681237+0000","last_fullsized":"2024-09-15T21:16:50.681237+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":23,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681198+0000","last_change":"2024-09-15T21:16:46.353967+0000","last_active":"2024-09-15T21:16:50.681198+0000","last_peered":"2024-09-15T21:16:50.681198+0000","last_clean":"2024-09-15T21:16:50.681198+0000","last_became_active":"2024-09-15T21:16:44.356586+0000","last_became_peered":"2024-09-15T21:16:44.356586+0000","last_unstale":"2024-09-15T21:16:50.681198+0000","last_undegraded":"2024-09-15T21:16:50.681198+0000","last_fullsized":"2024-09-15T21:16:50.681198+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.614015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":24,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.681049+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:50.681049+0000","last_peered":"2024-09-15T21:16:50.681049+0000","last_clean":"2024-09-15T21:16:50.681049+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:50.681049+0000","last_undegraded":"2024-09-15T21:16:50.681049+0000","last_fullsized":"2024-09-15T21:16:50.681049+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.677942+0000","last_change":"2024-09-15T21:16:46.355174+0000","last_active":"2024-09-15T21:16:50.677942+0000","last_peered":"2024-09-15T21:16:50.677942+0000","last_clean":"2024-09-15T21:16:50.677942+0000","last_became_active":"2024-09-15T21:16:44.348325+0000","last_became_peered":"2024-09-15T21:16:44.348325+0000","last_unstale":"2024-09-15T21:16:50.677942+0000","last_undegraded":"2024-09-15T21:16:50.677942+0000","last_fullsized":"2024-09-15T21:16:50.677942+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:08.222799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037593539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:16:50.682834+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:50.682834+0000","last_peered":"2024-09-15T21:16:50.682834+0000","last_clean":"2024-09-15T21:16:50.682834+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:50.682834+0000","last_undegraded":"2024-09-15T21:16:50.682834+0000","last_fullsized":"2024-09-15T21:16:50.682834+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":33188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":143360,"data_stored":33207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":143360,"data_stored":33207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:16:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-15T21:16:54.026 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-15T21:16:54.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-15T21:16:54.400 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'unfounddel_pool' created 2024-09-15T21:16:54.420 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-15T21:16:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-15T21:16:56.424 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:16:56.424 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:16:56.478 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:16:56.478 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-15T21:16:56.802 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:16:56.798+0000 7fefb254a640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-15T21:16:57.428 INFO:teuthology.orchestra.run.smithi114.stderr:marked down osd.2. 2024-09-15T21:16:57.453 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-15T21:16:57.769 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:16:57.766+0000 7fefb254a640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-15T21:16:58.436 INFO:teuthology.orchestra.run.smithi114.stderr:marked out osd.2. 2024-09-15T21:16:58.457 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-15T21:16:58.457 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-15T21:16:58.504 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:16:58.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:16:58.705 INFO:teuthology.orchestra.run.smithi114.stdout:34359738377 2024-09-15T21:16:58.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:16:58.712 INFO:teuthology.orchestra.run.smithi114.stdout:42949672969 2024-09-15T21:16:58.712 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:16:58.999 INFO:teuthology.orchestra.run.smithi114.stdout:34359738376 2024-09-15T21:16:59.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-15T21:16:59.050 INFO:teuthology.orchestra.run.smithi114.stdout:42949672968 2024-09-15T21:16:59.067 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672968 for osd.0 2024-09-15T21:17:00.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:00.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:00.330 INFO:teuthology.orchestra.run.smithi114.stdout:34359738376 2024-09-15T21:17:00.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-15T21:17:00.372 INFO:teuthology.orchestra.run.smithi114.stdout:42949672968 2024-09-15T21:17:00.390 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672968 for osd.0 2024-09-15T21:17:01.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:01.391 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:01.675 INFO:teuthology.orchestra.run.smithi114.stdout:34359738377 2024-09-15T21:17:01.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738377 for osd.1 2024-09-15T21:17:01.695 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:01.706 INFO:teuthology.orchestra.run.smithi114.stdout:42949672970 2024-09-15T21:17:01.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 42949672969 got 42949672970 for osd.0 2024-09-15T21:17:01.724 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:01.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-15T21:17:01.724 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:02.068 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:02.069 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:02.086 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-15T21:17:00.276440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54424,"kb_used_data":356,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689256,"statfs":{"total":193273528320,"available":193217798144,"internally_reserved":0,"allocated":364544,"data_stored":94164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001037"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.438386+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:58.438386+0000","last_peered":"2024-09-15T21:16:58.438386+0000","last_clean":"2024-09-15T21:16:58.438386+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:58.438386+0000","last_undegraded":"2024-09-15T21:16:58.438386+0000","last_fullsized":"2024-09-15T21:16:58.438386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438685+0000","last_change":"2024-09-15T21:16:57.426094+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438685+0000","last_undegraded":"2024-09-15T21:16:58.438685+0000","last_fullsized":"2024-09-15T21:16:58.438685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:16:57.424106+0000","last_change":"2024-09-15T21:16:57.424106+0000","last_active":"2024-09-15T21:16:57.424106+0000","last_peered":"2024-09-15T21:16:56.548389+0000","last_clean":"2024-09-15T21:16:56.548389+0000","last_became_active":"2024-09-15T21:16:55.405925+0000","last_became_peered":"2024-09-15T21:16:55.405925+0000","last_unstale":"2024-09-15T21:16:57.424106+0000","last_undegraded":"2024-09-15T21:16:57.424106+0000","last_fullsized":"2024-09-15T21:16:57.424106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438806+0000","last_change":"2024-09-15T21:16:57.427906+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438806+0000","last_undegraded":"2024-09-15T21:16:58.438806+0000","last_fullsized":"2024-09-15T21:16:58.438806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438367+0000","last_change":"2024-09-15T21:16:57.426190+0000","last_active":"2024-09-15T21:16:57.426190+0000","last_peered":"2024-09-15T21:16:56.406972+0000","last_clean":"2024-09-15T21:16:56.406972+0000","last_became_active":"2024-09-15T21:16:55.401718+0000","last_became_peered":"2024-09-15T21:16:55.401718+0000","last_unstale":"2024-09-15T21:16:58.438367+0000","last_undegraded":"2024-09-15T21:16:58.438367+0000","last_fullsized":"2024-09-15T21:16:58.438367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431577+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:58.431577+0000","last_peered":"2024-09-15T21:16:58.431577+0000","last_clean":"2024-09-15T21:16:58.431577+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:58.431577+0000","last_undegraded":"2024-09-15T21:16:58.431577+0000","last_fullsized":"2024-09-15T21:16:58.431577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.432295+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:58.432295+0000","last_peered":"2024-09-15T21:16:58.432295+0000","last_clean":"2024-09-15T21:16:58.432295+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:58.432295+0000","last_undegraded":"2024-09-15T21:16:58.432295+0000","last_fullsized":"2024-09-15T21:16:58.432295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.430329+0000","last_change":"2024-09-15T21:16:57.424848+0000","last_active":"2024-09-15T21:16:57.424848+0000","last_peered":"2024-09-15T21:16:56.405960+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:16:55.401751+0000","last_became_peered":"2024-09-15T21:16:55.401751+0000","last_unstale":"2024-09-15T21:16:58.430329+0000","last_undegraded":"2024-09-15T21:16:58.430329+0000","last_fullsized":"2024-09-15T21:16:58.430329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431120+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:58.431120+0000","last_peered":"2024-09-15T21:16:58.431120+0000","last_clean":"2024-09-15T21:16:58.431120+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:58.431120+0000","last_undegraded":"2024-09-15T21:16:58.431120+0000","last_fullsized":"2024-09-15T21:16:58.431120+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:16:57.423358+0000","last_change":"2024-09-15T21:16:57.423358+0000","last_active":"2024-09-15T21:16:57.423358+0000","last_peered":"2024-09-15T21:16:56.548350+0000","last_clean":"2024-09-15T21:16:56.548350+0000","last_became_active":"2024-09-15T21:16:55.395976+0000","last_became_peered":"2024-09-15T21:16:55.395976+0000","last_unstale":"2024-09-15T21:16:57.423358+0000","last_undegraded":"2024-09-15T21:16:57.423358+0000","last_fullsized":"2024-09-15T21:16:57.423358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.437029+0000","last_change":"2024-09-15T21:16:57.426440+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.437029+0000","last_undegraded":"2024-09-15T21:16:58.437029+0000","last_fullsized":"2024-09-15T21:16:58.437029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437999+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:58.437999+0000","last_peered":"2024-09-15T21:16:58.437999+0000","last_clean":"2024-09-15T21:16:58.437999+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:58.437999+0000","last_undegraded":"2024-09-15T21:16:58.437999+0000","last_fullsized":"2024-09-15T21:16:58.437999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437979+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:58.437979+0000","last_peered":"2024-09-15T21:16:58.437979+0000","last_clean":"2024-09-15T21:16:58.437979+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:58.437979+0000","last_undegraded":"2024-09-15T21:16:58.437979+0000","last_fullsized":"2024-09-15T21:16:58.437979+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.432062+0000","last_change":"2024-09-15T21:16:57.422360+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.432062+0000","last_undegraded":"2024-09-15T21:16:58.432062+0000","last_fullsized":"2024-09-15T21:16:58.432062+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":31,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:16:57.425152+0000","last_change":"2024-09-15T21:16:57.425152+0000","last_active":"2024-09-15T21:16:57.425152+0000","last_peered":"2024-09-15T21:16:56.408510+0000","last_clean":"2024-09-15T21:16:56.408510+0000","last_became_active":"2024-09-15T21:16:44.356586+0000","last_became_peered":"2024-09-15T21:16:44.356586+0000","last_unstale":"2024-09-15T21:16:57.425152+0000","last_undegraded":"2024-09-15T21:16:57.425152+0000","last_fullsized":"2024-09-15T21:16:57.425152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.430298+0000","last_change":"2024-09-15T21:16:57.422043+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.430298+0000","last_undegraded":"2024-09-15T21:16:58.430298+0000","last_fullsized":"2024-09-15T21:16:58.430298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431527+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:58.431527+0000","last_peered":"2024-09-15T21:16:58.431527+0000","last_clean":"2024-09-15T21:16:58.431527+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:58.431527+0000","last_undegraded":"2024-09-15T21:16:58.431527+0000","last_fullsized":"2024-09-15T21:16:58.431527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431507+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:58.431507+0000","last_peered":"2024-09-15T21:16:58.431507+0000","last_clean":"2024-09-15T21:16:58.431507+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:58.431507+0000","last_undegraded":"2024-09-15T21:16:58.431507+0000","last_fullsized":"2024-09-15T21:16:58.431507+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431518+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:58.431518+0000","last_peered":"2024-09-15T21:16:58.431518+0000","last_clean":"2024-09-15T21:16:58.431518+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:58.431518+0000","last_undegraded":"2024-09-15T21:16:58.431518+0000","last_fullsized":"2024-09-15T21:16:58.431518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438653+0000","last_change":"2024-09-15T21:16:57.426281+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438653+0000","last_undegraded":"2024-09-15T21:16:58.438653+0000","last_fullsized":"2024-09-15T21:16:58.438653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.436097+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:58.436097+0000","last_peered":"2024-09-15T21:16:58.436097+0000","last_clean":"2024-09-15T21:16:58.436097+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:58.436097+0000","last_undegraded":"2024-09-15T21:16:58.436097+0000","last_fullsized":"2024-09-15T21:16:58.436097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431763+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:58.431763+0000","last_peered":"2024-09-15T21:16:58.431763+0000","last_clean":"2024-09-15T21:16:58.431763+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:58.431763+0000","last_undegraded":"2024-09-15T21:16:58.431763+0000","last_fullsized":"2024-09-15T21:16:58.431763+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:16:57.427208+0000","last_change":"2024-09-15T21:16:57.427208+0000","last_active":"2024-09-15T21:16:57.427208+0000","last_peered":"2024-09-15T21:16:56.409498+0000","last_clean":"2024-09-15T21:16:56.409498+0000","last_became_active":"2024-09-15T21:16:44.351500+0000","last_became_peered":"2024-09-15T21:16:44.351500+0000","last_unstale":"2024-09-15T21:16:57.427208+0000","last_undegraded":"2024-09-15T21:16:57.427208+0000","last_fullsized":"2024-09-15T21:16:57.427208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.431333+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:58.431333+0000","last_peered":"2024-09-15T21:16:58.431333+0000","last_clean":"2024-09-15T21:16:58.431333+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:58.431333+0000","last_undegraded":"2024-09-15T21:16:58.431333+0000","last_fullsized":"2024-09-15T21:16:58.431333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":48778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":45386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:02.087 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:02.365 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:02.365 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:02.382 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-15T21:17:02.276927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54440,"kb_used_data":376,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689240,"statfs":{"total":193273528320,"available":193217781760,"internally_reserved":0,"allocated":385024,"data_stored":100273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873562"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.438386+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:58.438386+0000","last_peered":"2024-09-15T21:16:58.438386+0000","last_clean":"2024-09-15T21:16:58.438386+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:58.438386+0000","last_undegraded":"2024-09-15T21:16:58.438386+0000","last_fullsized":"2024-09-15T21:16:58.438386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438685+0000","last_change":"2024-09-15T21:16:57.426094+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438685+0000","last_undegraded":"2024-09-15T21:16:58.438685+0000","last_fullsized":"2024-09-15T21:16:58.438685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.452843+0000","last_change":"2024-09-15T21:16:59.452558+0000","last_active":"2024-09-15T21:16:59.452843+0000","last_peered":"2024-09-15T21:16:59.452843+0000","last_clean":"2024-09-15T21:16:59.452843+0000","last_became_active":"2024-09-15T21:16:59.451802+0000","last_became_peered":"2024-09-15T21:16:59.451802+0000","last_unstale":"2024-09-15T21:16:59.452843+0000","last_undegraded":"2024-09-15T21:16:59.452843+0000","last_fullsized":"2024-09-15T21:16:59.452843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:19.006881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438806+0000","last_change":"2024-09-15T21:16:57.427906+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438806+0000","last_undegraded":"2024-09-15T21:16:58.438806+0000","last_fullsized":"2024-09-15T21:16:58.438806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438367+0000","last_change":"2024-09-15T21:16:57.426190+0000","last_active":"2024-09-15T21:16:57.426190+0000","last_peered":"2024-09-15T21:16:56.406972+0000","last_clean":"2024-09-15T21:16:56.406972+0000","last_became_active":"2024-09-15T21:16:55.401718+0000","last_became_peered":"2024-09-15T21:16:55.401718+0000","last_unstale":"2024-09-15T21:16:58.438367+0000","last_undegraded":"2024-09-15T21:16:58.438367+0000","last_fullsized":"2024-09-15T21:16:58.438367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438948+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:59.438948+0000","last_peered":"2024-09-15T21:16:59.438948+0000","last_clean":"2024-09-15T21:16:59.438948+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:59.438948+0000","last_undegraded":"2024-09-15T21:16:59.438948+0000","last_fullsized":"2024-09-15T21:16:59.438948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438682+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:59.438682+0000","last_peered":"2024-09-15T21:16:59.438682+0000","last_clean":"2024-09-15T21:16:59.438682+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:59.438682+0000","last_undegraded":"2024-09-15T21:16:59.438682+0000","last_fullsized":"2024-09-15T21:16:59.438682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.430329+0000","last_change":"2024-09-15T21:16:57.424848+0000","last_active":"2024-09-15T21:16:57.424848+0000","last_peered":"2024-09-15T21:16:56.405960+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:16:55.401751+0000","last_became_peered":"2024-09-15T21:16:55.401751+0000","last_unstale":"2024-09-15T21:16:58.430329+0000","last_undegraded":"2024-09-15T21:16:58.430329+0000","last_fullsized":"2024-09-15T21:16:58.430329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438635+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:59.438635+0000","last_peered":"2024-09-15T21:16:59.438635+0000","last_clean":"2024-09-15T21:16:59.438635+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:59.438635+0000","last_undegraded":"2024-09-15T21:16:59.438635+0000","last_fullsized":"2024-09-15T21:16:59.438635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.451171+0000","last_change":"2024-09-15T21:16:59.450931+0000","last_active":"2024-09-15T21:16:59.451171+0000","last_peered":"2024-09-15T21:16:59.451171+0000","last_clean":"2024-09-15T21:16:59.451171+0000","last_became_active":"2024-09-15T21:16:59.450116+0000","last_became_peered":"2024-09-15T21:16:59.450116+0000","last_unstale":"2024-09-15T21:16:59.451171+0000","last_undegraded":"2024-09-15T21:16:59.451171+0000","last_fullsized":"2024-09-15T21:16:59.451171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:35.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.437029+0000","last_change":"2024-09-15T21:16:57.426440+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.437029+0000","last_undegraded":"2024-09-15T21:16:58.437029+0000","last_fullsized":"2024-09-15T21:16:58.437029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437999+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:58.437999+0000","last_peered":"2024-09-15T21:16:58.437999+0000","last_clean":"2024-09-15T21:16:58.437999+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:58.437999+0000","last_undegraded":"2024-09-15T21:16:58.437999+0000","last_fullsized":"2024-09-15T21:16:58.437999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437979+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:58.437979+0000","last_peered":"2024-09-15T21:16:58.437979+0000","last_clean":"2024-09-15T21:16:58.437979+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:58.437979+0000","last_undegraded":"2024-09-15T21:16:58.437979+0000","last_fullsized":"2024-09-15T21:16:58.437979+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.449422+0000","last_change":"2024-09-15T21:16:59.449193+0000","last_active":"2024-09-15T21:16:59.449422+0000","last_peered":"2024-09-15T21:16:59.449422+0000","last_clean":"2024-09-15T21:16:59.449422+0000","last_became_active":"2024-09-15T21:16:59.448448+0000","last_became_peered":"2024-09-15T21:16:59.448448+0000","last_unstale":"2024-09-15T21:16:59.449422+0000","last_undegraded":"2024-09-15T21:16:59.449422+0000","last_fullsized":"2024-09-15T21:16:59.449422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:56.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.458409+0000","last_change":"2024-09-15T21:16:59.458213+0000","last_active":"2024-09-15T21:16:59.458409+0000","last_peered":"2024-09-15T21:16:59.458409+0000","last_clean":"2024-09-15T21:16:59.458409+0000","last_became_active":"2024-09-15T21:16:59.453118+0000","last_became_peered":"2024-09-15T21:16:59.453118+0000","last_unstale":"2024-09-15T21:16:59.458409+0000","last_undegraded":"2024-09-15T21:16:59.458409+0000","last_fullsized":"2024-09-15T21:16:59.458409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:03.427621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446971+0000","last_change":"2024-09-15T21:16:59.446702+0000","last_active":"2024-09-15T21:16:59.446971+0000","last_peered":"2024-09-15T21:16:59.446971+0000","last_clean":"2024-09-15T21:16:59.446971+0000","last_became_active":"2024-09-15T21:16:59.446093+0000","last_became_peered":"2024-09-15T21:16:59.446093+0000","last_unstale":"2024-09-15T21:16:59.446971+0000","last_undegraded":"2024-09-15T21:16:59.446971+0000","last_fullsized":"2024-09-15T21:16:59.446971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:13.220833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440780+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:59.440780+0000","last_peered":"2024-09-15T21:16:59.440780+0000","last_clean":"2024-09-15T21:16:59.440780+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:59.440780+0000","last_undegraded":"2024-09-15T21:16:59.440780+0000","last_fullsized":"2024-09-15T21:16:59.440780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439273+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:59.439273+0000","last_peered":"2024-09-15T21:16:59.439273+0000","last_clean":"2024-09-15T21:16:59.439273+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:59.439273+0000","last_undegraded":"2024-09-15T21:16:59.439273+0000","last_fullsized":"2024-09-15T21:16:59.439273+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440552+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:59.440552+0000","last_peered":"2024-09-15T21:16:59.440552+0000","last_clean":"2024-09-15T21:16:59.440552+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:59.440552+0000","last_undegraded":"2024-09-15T21:16:59.440552+0000","last_fullsized":"2024-09-15T21:16:59.440552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438653+0000","last_change":"2024-09-15T21:16:57.426281+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438653+0000","last_undegraded":"2024-09-15T21:16:58.438653+0000","last_fullsized":"2024-09-15T21:16:58.438653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439912+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:59.439912+0000","last_peered":"2024-09-15T21:16:59.439912+0000","last_clean":"2024-09-15T21:16:59.439912+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:59.439912+0000","last_undegraded":"2024-09-15T21:16:59.439912+0000","last_fullsized":"2024-09-15T21:16:59.439912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440743+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:59.440743+0000","last_peered":"2024-09-15T21:16:59.440743+0000","last_clean":"2024-09-15T21:16:59.440743+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:59.440743+0000","last_undegraded":"2024-09-15T21:16:59.440743+0000","last_fullsized":"2024-09-15T21:16:59.440743+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.453189+0000","last_change":"2024-09-15T21:16:59.452491+0000","last_active":"2024-09-15T21:16:59.453189+0000","last_peered":"2024-09-15T21:16:59.453189+0000","last_clean":"2024-09-15T21:16:59.453189+0000","last_became_active":"2024-09-15T21:16:59.451619+0000","last_became_peered":"2024-09-15T21:16:59.451619+0000","last_unstale":"2024-09-15T21:16:59.453189+0000","last_undegraded":"2024-09-15T21:16:59.453189+0000","last_fullsized":"2024-09-15T21:16:59.453189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:31.903673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438957+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:59.438957+0000","last_peered":"2024-09-15T21:16:59.438957+0000","last_clean":"2024-09-15T21:16:59.438957+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:59.438957+0000","last_undegraded":"2024-09-15T21:16:59.438957+0000","last_fullsized":"2024-09-15T21:16:59.438957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":48778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:02.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:02.673 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:02.690 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-15T21:17:02.276927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54440,"kb_used_data":376,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689240,"statfs":{"total":193273528320,"available":193217781760,"internally_reserved":0,"allocated":385024,"data_stored":100273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.873562"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.438386+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:58.438386+0000","last_peered":"2024-09-15T21:16:58.438386+0000","last_clean":"2024-09-15T21:16:58.438386+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:58.438386+0000","last_undegraded":"2024-09-15T21:16:58.438386+0000","last_fullsized":"2024-09-15T21:16:58.438386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438685+0000","last_change":"2024-09-15T21:16:57.426094+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438685+0000","last_undegraded":"2024-09-15T21:16:58.438685+0000","last_fullsized":"2024-09-15T21:16:58.438685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.452843+0000","last_change":"2024-09-15T21:16:59.452558+0000","last_active":"2024-09-15T21:16:59.452843+0000","last_peered":"2024-09-15T21:16:59.452843+0000","last_clean":"2024-09-15T21:16:59.452843+0000","last_became_active":"2024-09-15T21:16:59.451802+0000","last_became_peered":"2024-09-15T21:16:59.451802+0000","last_unstale":"2024-09-15T21:16:59.452843+0000","last_undegraded":"2024-09-15T21:16:59.452843+0000","last_fullsized":"2024-09-15T21:16:59.452843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:19.006881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438806+0000","last_change":"2024-09-15T21:16:57.427906+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438806+0000","last_undegraded":"2024-09-15T21:16:58.438806+0000","last_fullsized":"2024-09-15T21:16:58.438806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438367+0000","last_change":"2024-09-15T21:16:57.426190+0000","last_active":"2024-09-15T21:16:57.426190+0000","last_peered":"2024-09-15T21:16:56.406972+0000","last_clean":"2024-09-15T21:16:56.406972+0000","last_became_active":"2024-09-15T21:16:55.401718+0000","last_became_peered":"2024-09-15T21:16:55.401718+0000","last_unstale":"2024-09-15T21:16:58.438367+0000","last_undegraded":"2024-09-15T21:16:58.438367+0000","last_fullsized":"2024-09-15T21:16:58.438367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438948+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:59.438948+0000","last_peered":"2024-09-15T21:16:59.438948+0000","last_clean":"2024-09-15T21:16:59.438948+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:59.438948+0000","last_undegraded":"2024-09-15T21:16:59.438948+0000","last_fullsized":"2024-09-15T21:16:59.438948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438682+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:59.438682+0000","last_peered":"2024-09-15T21:16:59.438682+0000","last_clean":"2024-09-15T21:16:59.438682+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:59.438682+0000","last_undegraded":"2024-09-15T21:16:59.438682+0000","last_fullsized":"2024-09-15T21:16:59.438682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.430329+0000","last_change":"2024-09-15T21:16:57.424848+0000","last_active":"2024-09-15T21:16:57.424848+0000","last_peered":"2024-09-15T21:16:56.405960+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:16:55.401751+0000","last_became_peered":"2024-09-15T21:16:55.401751+0000","last_unstale":"2024-09-15T21:16:58.430329+0000","last_undegraded":"2024-09-15T21:16:58.430329+0000","last_fullsized":"2024-09-15T21:16:58.430329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438635+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:59.438635+0000","last_peered":"2024-09-15T21:16:59.438635+0000","last_clean":"2024-09-15T21:16:59.438635+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:59.438635+0000","last_undegraded":"2024-09-15T21:16:59.438635+0000","last_fullsized":"2024-09-15T21:16:59.438635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.451171+0000","last_change":"2024-09-15T21:16:59.450931+0000","last_active":"2024-09-15T21:16:59.451171+0000","last_peered":"2024-09-15T21:16:59.451171+0000","last_clean":"2024-09-15T21:16:59.451171+0000","last_became_active":"2024-09-15T21:16:59.450116+0000","last_became_peered":"2024-09-15T21:16:59.450116+0000","last_unstale":"2024-09-15T21:16:59.451171+0000","last_undegraded":"2024-09-15T21:16:59.451171+0000","last_fullsized":"2024-09-15T21:16:59.451171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:35.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.437029+0000","last_change":"2024-09-15T21:16:57.426440+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.437029+0000","last_undegraded":"2024-09-15T21:16:58.437029+0000","last_fullsized":"2024-09-15T21:16:58.437029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437999+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:58.437999+0000","last_peered":"2024-09-15T21:16:58.437999+0000","last_clean":"2024-09-15T21:16:58.437999+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:58.437999+0000","last_undegraded":"2024-09-15T21:16:58.437999+0000","last_fullsized":"2024-09-15T21:16:58.437999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437979+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:58.437979+0000","last_peered":"2024-09-15T21:16:58.437979+0000","last_clean":"2024-09-15T21:16:58.437979+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:58.437979+0000","last_undegraded":"2024-09-15T21:16:58.437979+0000","last_fullsized":"2024-09-15T21:16:58.437979+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.449422+0000","last_change":"2024-09-15T21:16:59.449193+0000","last_active":"2024-09-15T21:16:59.449422+0000","last_peered":"2024-09-15T21:16:59.449422+0000","last_clean":"2024-09-15T21:16:59.449422+0000","last_became_active":"2024-09-15T21:16:59.448448+0000","last_became_peered":"2024-09-15T21:16:59.448448+0000","last_unstale":"2024-09-15T21:16:59.449422+0000","last_undegraded":"2024-09-15T21:16:59.449422+0000","last_fullsized":"2024-09-15T21:16:59.449422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:56.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.458409+0000","last_change":"2024-09-15T21:16:59.458213+0000","last_active":"2024-09-15T21:16:59.458409+0000","last_peered":"2024-09-15T21:16:59.458409+0000","last_clean":"2024-09-15T21:16:59.458409+0000","last_became_active":"2024-09-15T21:16:59.453118+0000","last_became_peered":"2024-09-15T21:16:59.453118+0000","last_unstale":"2024-09-15T21:16:59.458409+0000","last_undegraded":"2024-09-15T21:16:59.458409+0000","last_fullsized":"2024-09-15T21:16:59.458409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:03.427621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446971+0000","last_change":"2024-09-15T21:16:59.446702+0000","last_active":"2024-09-15T21:16:59.446971+0000","last_peered":"2024-09-15T21:16:59.446971+0000","last_clean":"2024-09-15T21:16:59.446971+0000","last_became_active":"2024-09-15T21:16:59.446093+0000","last_became_peered":"2024-09-15T21:16:59.446093+0000","last_unstale":"2024-09-15T21:16:59.446971+0000","last_undegraded":"2024-09-15T21:16:59.446971+0000","last_fullsized":"2024-09-15T21:16:59.446971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:13.220833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440780+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:59.440780+0000","last_peered":"2024-09-15T21:16:59.440780+0000","last_clean":"2024-09-15T21:16:59.440780+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:59.440780+0000","last_undegraded":"2024-09-15T21:16:59.440780+0000","last_fullsized":"2024-09-15T21:16:59.440780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439273+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:59.439273+0000","last_peered":"2024-09-15T21:16:59.439273+0000","last_clean":"2024-09-15T21:16:59.439273+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:59.439273+0000","last_undegraded":"2024-09-15T21:16:59.439273+0000","last_fullsized":"2024-09-15T21:16:59.439273+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440552+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:59.440552+0000","last_peered":"2024-09-15T21:16:59.440552+0000","last_clean":"2024-09-15T21:16:59.440552+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:59.440552+0000","last_undegraded":"2024-09-15T21:16:59.440552+0000","last_fullsized":"2024-09-15T21:16:59.440552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438653+0000","last_change":"2024-09-15T21:16:57.426281+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438653+0000","last_undegraded":"2024-09-15T21:16:58.438653+0000","last_fullsized":"2024-09-15T21:16:58.438653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439912+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:59.439912+0000","last_peered":"2024-09-15T21:16:59.439912+0000","last_clean":"2024-09-15T21:16:59.439912+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:59.439912+0000","last_undegraded":"2024-09-15T21:16:59.439912+0000","last_fullsized":"2024-09-15T21:16:59.439912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440743+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:59.440743+0000","last_peered":"2024-09-15T21:16:59.440743+0000","last_clean":"2024-09-15T21:16:59.440743+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:59.440743+0000","last_undegraded":"2024-09-15T21:16:59.440743+0000","last_fullsized":"2024-09-15T21:16:59.440743+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.453189+0000","last_change":"2024-09-15T21:16:59.452491+0000","last_active":"2024-09-15T21:16:59.453189+0000","last_peered":"2024-09-15T21:16:59.453189+0000","last_clean":"2024-09-15T21:16:59.453189+0000","last_became_active":"2024-09-15T21:16:59.451619+0000","last_became_peered":"2024-09-15T21:16:59.451619+0000","last_unstale":"2024-09-15T21:16:59.453189+0000","last_undegraded":"2024-09-15T21:16:59.453189+0000","last_fullsized":"2024-09-15T21:16:59.453189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:31.903673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438957+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:59.438957+0000","last_peered":"2024-09-15T21:16:59.438957+0000","last_clean":"2024-09-15T21:16:59.438957+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:59.438957+0000","last_undegraded":"2024-09-15T21:16:59.438957+0000","last_fullsized":"2024-09-15T21:16:59.438957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":48778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:05.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:06.011 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:06.012 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:06.030 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-15T21:17:04.277490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54440,"kb_used_data":376,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689240,"statfs":{"total":193273528320,"available":193217781760,"internally_reserved":0,"allocated":385024,"data_stored":100273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.857590"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.438386+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:58.438386+0000","last_peered":"2024-09-15T21:16:58.438386+0000","last_clean":"2024-09-15T21:16:58.438386+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:58.438386+0000","last_undegraded":"2024-09-15T21:16:58.438386+0000","last_fullsized":"2024-09-15T21:16:58.438386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438685+0000","last_change":"2024-09-15T21:16:57.426094+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438685+0000","last_undegraded":"2024-09-15T21:16:58.438685+0000","last_fullsized":"2024-09-15T21:16:58.438685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.452843+0000","last_change":"2024-09-15T21:16:59.452558+0000","last_active":"2024-09-15T21:16:59.452843+0000","last_peered":"2024-09-15T21:16:59.452843+0000","last_clean":"2024-09-15T21:16:59.452843+0000","last_became_active":"2024-09-15T21:16:59.451802+0000","last_became_peered":"2024-09-15T21:16:59.451802+0000","last_unstale":"2024-09-15T21:16:59.452843+0000","last_undegraded":"2024-09-15T21:16:59.452843+0000","last_fullsized":"2024-09-15T21:16:59.452843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:19.006881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438806+0000","last_change":"2024-09-15T21:16:57.427906+0000","last_active":"2024-09-15T21:16:54.375167+0000","last_peered":"2024-09-15T21:16:54.375167+0000","last_clean":"2024-09-15T21:16:54.375167+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438806+0000","last_undegraded":"2024-09-15T21:16:58.438806+0000","last_fullsized":"2024-09-15T21:16:58.438806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438367+0000","last_change":"2024-09-15T21:16:57.426190+0000","last_active":"2024-09-15T21:16:57.426190+0000","last_peered":"2024-09-15T21:16:56.406972+0000","last_clean":"2024-09-15T21:16:56.406972+0000","last_became_active":"2024-09-15T21:16:55.401718+0000","last_became_peered":"2024-09-15T21:16:55.401718+0000","last_unstale":"2024-09-15T21:16:58.438367+0000","last_undegraded":"2024-09-15T21:16:58.438367+0000","last_fullsized":"2024-09-15T21:16:58.438367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438948+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:59.438948+0000","last_peered":"2024-09-15T21:16:59.438948+0000","last_clean":"2024-09-15T21:16:59.438948+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:59.438948+0000","last_undegraded":"2024-09-15T21:16:59.438948+0000","last_fullsized":"2024-09-15T21:16:59.438948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438682+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:59.438682+0000","last_peered":"2024-09-15T21:16:59.438682+0000","last_clean":"2024-09-15T21:16:59.438682+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:59.438682+0000","last_undegraded":"2024-09-15T21:16:59.438682+0000","last_fullsized":"2024-09-15T21:16:59.438682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.430329+0000","last_change":"2024-09-15T21:16:57.424848+0000","last_active":"2024-09-15T21:16:57.424848+0000","last_peered":"2024-09-15T21:16:56.405960+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:16:55.401751+0000","last_became_peered":"2024-09-15T21:16:55.401751+0000","last_unstale":"2024-09-15T21:16:58.430329+0000","last_undegraded":"2024-09-15T21:16:58.430329+0000","last_fullsized":"2024-09-15T21:16:58.430329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438635+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:59.438635+0000","last_peered":"2024-09-15T21:16:59.438635+0000","last_clean":"2024-09-15T21:16:59.438635+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:59.438635+0000","last_undegraded":"2024-09-15T21:16:59.438635+0000","last_fullsized":"2024-09-15T21:16:59.438635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.451171+0000","last_change":"2024-09-15T21:16:59.450931+0000","last_active":"2024-09-15T21:16:59.451171+0000","last_peered":"2024-09-15T21:16:59.451171+0000","last_clean":"2024-09-15T21:16:59.451171+0000","last_became_active":"2024-09-15T21:16:59.450116+0000","last_became_peered":"2024-09-15T21:16:59.450116+0000","last_unstale":"2024-09-15T21:16:59.451171+0000","last_undegraded":"2024-09-15T21:16:59.451171+0000","last_fullsized":"2024-09-15T21:16:59.451171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:35.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.437029+0000","last_change":"2024-09-15T21:16:57.426440+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.437029+0000","last_undegraded":"2024-09-15T21:16:58.437029+0000","last_fullsized":"2024-09-15T21:16:58.437029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437999+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:58.437999+0000","last_peered":"2024-09-15T21:16:58.437999+0000","last_clean":"2024-09-15T21:16:58.437999+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:58.437999+0000","last_undegraded":"2024-09-15T21:16:58.437999+0000","last_fullsized":"2024-09-15T21:16:58.437999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:16:58.437979+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:58.437979+0000","last_peered":"2024-09-15T21:16:58.437979+0000","last_clean":"2024-09-15T21:16:58.437979+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:58.437979+0000","last_undegraded":"2024-09-15T21:16:58.437979+0000","last_fullsized":"2024-09-15T21:16:58.437979+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.449422+0000","last_change":"2024-09-15T21:16:59.449193+0000","last_active":"2024-09-15T21:16:59.449422+0000","last_peered":"2024-09-15T21:16:59.449422+0000","last_clean":"2024-09-15T21:16:59.449422+0000","last_became_active":"2024-09-15T21:16:59.448448+0000","last_became_peered":"2024-09-15T21:16:59.448448+0000","last_unstale":"2024-09-15T21:16:59.449422+0000","last_undegraded":"2024-09-15T21:16:59.449422+0000","last_fullsized":"2024-09-15T21:16:59.449422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:56.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.458409+0000","last_change":"2024-09-15T21:16:59.458213+0000","last_active":"2024-09-15T21:16:59.458409+0000","last_peered":"2024-09-15T21:16:59.458409+0000","last_clean":"2024-09-15T21:16:59.458409+0000","last_became_active":"2024-09-15T21:16:59.453118+0000","last_became_peered":"2024-09-15T21:16:59.453118+0000","last_unstale":"2024-09-15T21:16:59.458409+0000","last_undegraded":"2024-09-15T21:16:59.458409+0000","last_fullsized":"2024-09-15T21:16:59.458409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:03.427621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446971+0000","last_change":"2024-09-15T21:16:59.446702+0000","last_active":"2024-09-15T21:16:59.446971+0000","last_peered":"2024-09-15T21:16:59.446971+0000","last_clean":"2024-09-15T21:16:59.446971+0000","last_became_active":"2024-09-15T21:16:59.446093+0000","last_became_peered":"2024-09-15T21:16:59.446093+0000","last_unstale":"2024-09-15T21:16:59.446971+0000","last_undegraded":"2024-09-15T21:16:59.446971+0000","last_fullsized":"2024-09-15T21:16:59.446971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:13.220833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440780+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:59.440780+0000","last_peered":"2024-09-15T21:16:59.440780+0000","last_clean":"2024-09-15T21:16:59.440780+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:59.440780+0000","last_undegraded":"2024-09-15T21:16:59.440780+0000","last_fullsized":"2024-09-15T21:16:59.440780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439273+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:59.439273+0000","last_peered":"2024-09-15T21:16:59.439273+0000","last_clean":"2024-09-15T21:16:59.439273+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:59.439273+0000","last_undegraded":"2024-09-15T21:16:59.439273+0000","last_fullsized":"2024-09-15T21:16:59.439273+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440552+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:59.440552+0000","last_peered":"2024-09-15T21:16:59.440552+0000","last_clean":"2024-09-15T21:16:59.440552+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:59.440552+0000","last_undegraded":"2024-09-15T21:16:59.440552+0000","last_fullsized":"2024-09-15T21:16:59.440552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":21,"state":"peering","last_fresh":"2024-09-15T21:16:58.438653+0000","last_change":"2024-09-15T21:16:57.426281+0000","last_active":"2024-09-15T21:16:43.325385+0000","last_peered":"2024-09-15T21:16:43.325385+0000","last_clean":"2024-09-15T21:16:43.325385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:16:58.438653+0000","last_undegraded":"2024-09-15T21:16:58.438653+0000","last_fullsized":"2024-09-15T21:16:58.438653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439912+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:59.439912+0000","last_peered":"2024-09-15T21:16:59.439912+0000","last_clean":"2024-09-15T21:16:59.439912+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:59.439912+0000","last_undegraded":"2024-09-15T21:16:59.439912+0000","last_fullsized":"2024-09-15T21:16:59.439912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440743+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:59.440743+0000","last_peered":"2024-09-15T21:16:59.440743+0000","last_clean":"2024-09-15T21:16:59.440743+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:59.440743+0000","last_undegraded":"2024-09-15T21:16:59.440743+0000","last_fullsized":"2024-09-15T21:16:59.440743+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.453189+0000","last_change":"2024-09-15T21:16:59.452491+0000","last_active":"2024-09-15T21:16:59.453189+0000","last_peered":"2024-09-15T21:16:59.453189+0000","last_clean":"2024-09-15T21:16:59.453189+0000","last_became_active":"2024-09-15T21:16:59.451619+0000","last_became_peered":"2024-09-15T21:16:59.451619+0000","last_unstale":"2024-09-15T21:16:59.453189+0000","last_undegraded":"2024-09-15T21:16:59.453189+0000","last_fullsized":"2024-09-15T21:16:59.453189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:31.903673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438957+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:59.438957+0000","last_peered":"2024-09-15T21:16:59.438957+0000","last_clean":"2024-09-15T21:16:59.438957+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:59.438957+0000","last_undegraded":"2024-09-15T21:16:59.438957+0000","last_fullsized":"2024-09-15T21:16:59.438957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":48778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:06.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:06.319 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:06.319 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:06.336 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-15T21:17:06.277981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54452,"kb_used_data":384,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689228,"statfs":{"total":193273528320,"available":193217769472,"internally_reserved":0,"allocated":393216,"data_stored":102990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001972"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.435209+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:59.435209+0000","last_peered":"2024-09-15T21:16:59.435209+0000","last_clean":"2024-09-15T21:16:59.435209+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:59.435209+0000","last_undegraded":"2024-09-15T21:16:59.435209+0000","last_fullsized":"2024-09-15T21:16:59.435209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.447520+0000","last_change":"2024-09-15T21:16:59.445579+0000","last_active":"2024-09-15T21:16:59.447520+0000","last_peered":"2024-09-15T21:16:59.447520+0000","last_clean":"2024-09-15T21:16:59.447520+0000","last_became_active":"2024-09-15T21:16:59.444712+0000","last_became_peered":"2024-09-15T21:16:59.444712+0000","last_unstale":"2024-09-15T21:16:59.447520+0000","last_undegraded":"2024-09-15T21:16:59.447520+0000","last_fullsized":"2024-09-15T21:16:59.447520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:08:09.647705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.452843+0000","last_change":"2024-09-15T21:16:59.452558+0000","last_active":"2024-09-15T21:16:59.452843+0000","last_peered":"2024-09-15T21:16:59.452843+0000","last_clean":"2024-09-15T21:16:59.452843+0000","last_became_active":"2024-09-15T21:16:59.451802+0000","last_became_peered":"2024-09-15T21:16:59.451802+0000","last_unstale":"2024-09-15T21:16:59.452843+0000","last_undegraded":"2024-09-15T21:16:59.452843+0000","last_fullsized":"2024-09-15T21:16:59.452843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:19.006881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445384+0000","last_change":"2024-09-15T21:16:59.445105+0000","last_active":"2024-09-15T21:16:59.445384+0000","last_peered":"2024-09-15T21:16:59.445384+0000","last_clean":"2024-09-15T21:16:59.445384+0000","last_became_active":"2024-09-15T21:16:59.443707+0000","last_became_peered":"2024-09-15T21:16:59.443707+0000","last_unstale":"2024-09-15T21:16:59.445384+0000","last_undegraded":"2024-09-15T21:16:59.445384+0000","last_fullsized":"2024-09-15T21:16:59.445384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:00.666408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445850+0000","last_change":"2024-09-15T21:16:59.445580+0000","last_active":"2024-09-15T21:16:59.445850+0000","last_peered":"2024-09-15T21:16:59.445850+0000","last_clean":"2024-09-15T21:16:59.445850+0000","last_became_active":"2024-09-15T21:16:59.444782+0000","last_became_peered":"2024-09-15T21:16:59.444782+0000","last_unstale":"2024-09-15T21:16:59.445850+0000","last_undegraded":"2024-09-15T21:16:59.445850+0000","last_fullsized":"2024-09-15T21:16:59.445850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:26.734939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438948+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:59.438948+0000","last_peered":"2024-09-15T21:16:59.438948+0000","last_clean":"2024-09-15T21:16:59.438948+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:59.438948+0000","last_undegraded":"2024-09-15T21:16:59.438948+0000","last_fullsized":"2024-09-15T21:16:59.438948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438682+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:59.438682+0000","last_peered":"2024-09-15T21:16:59.438682+0000","last_clean":"2024-09-15T21:16:59.438682+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:59.438682+0000","last_undegraded":"2024-09-15T21:16:59.438682+0000","last_fullsized":"2024-09-15T21:16:59.438682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.448604+0000","last_change":"2024-09-15T21:16:59.448341+0000","last_active":"2024-09-15T21:16:59.448604+0000","last_peered":"2024-09-15T21:16:59.448604+0000","last_clean":"2024-09-15T21:16:59.448604+0000","last_became_active":"2024-09-15T21:16:59.447613+0000","last_became_peered":"2024-09-15T21:16:59.447613+0000","last_unstale":"2024-09-15T21:16:59.448604+0000","last_undegraded":"2024-09-15T21:16:59.448604+0000","last_fullsized":"2024-09-15T21:16:59.448604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:23.797403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438635+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:59.438635+0000","last_peered":"2024-09-15T21:16:59.438635+0000","last_clean":"2024-09-15T21:16:59.438635+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:59.438635+0000","last_undegraded":"2024-09-15T21:16:59.438635+0000","last_fullsized":"2024-09-15T21:16:59.438635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.451171+0000","last_change":"2024-09-15T21:16:59.450931+0000","last_active":"2024-09-15T21:16:59.451171+0000","last_peered":"2024-09-15T21:16:59.451171+0000","last_clean":"2024-09-15T21:16:59.451171+0000","last_became_active":"2024-09-15T21:16:59.450116+0000","last_became_peered":"2024-09-15T21:16:59.450116+0000","last_unstale":"2024-09-15T21:16:59.451171+0000","last_undegraded":"2024-09-15T21:16:59.451171+0000","last_fullsized":"2024-09-15T21:16:59.451171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:35.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445580+0000","last_change":"2024-09-15T21:16:59.445350+0000","last_active":"2024-09-15T21:16:59.445580+0000","last_peered":"2024-09-15T21:16:59.445580+0000","last_clean":"2024-09-15T21:16:59.445580+0000","last_became_active":"2024-09-15T21:16:59.443592+0000","last_became_peered":"2024-09-15T21:16:59.443592+0000","last_unstale":"2024-09-15T21:16:59.445580+0000","last_undegraded":"2024-09-15T21:16:59.445580+0000","last_fullsized":"2024-09-15T21:16:59.445580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.251696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.434158+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:59.434158+0000","last_peered":"2024-09-15T21:16:59.434158+0000","last_clean":"2024-09-15T21:16:59.434158+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:59.434158+0000","last_undegraded":"2024-09-15T21:16:59.434158+0000","last_fullsized":"2024-09-15T21:16:59.434158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.434425+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:59.434425+0000","last_peered":"2024-09-15T21:16:59.434425+0000","last_clean":"2024-09-15T21:16:59.434425+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:59.434425+0000","last_undegraded":"2024-09-15T21:16:59.434425+0000","last_fullsized":"2024-09-15T21:16:59.434425+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.449422+0000","last_change":"2024-09-15T21:16:59.449193+0000","last_active":"2024-09-15T21:16:59.449422+0000","last_peered":"2024-09-15T21:16:59.449422+0000","last_clean":"2024-09-15T21:16:59.449422+0000","last_became_active":"2024-09-15T21:16:59.448448+0000","last_became_peered":"2024-09-15T21:16:59.448448+0000","last_unstale":"2024-09-15T21:16:59.449422+0000","last_undegraded":"2024-09-15T21:16:59.449422+0000","last_fullsized":"2024-09-15T21:16:59.449422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:56.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.458409+0000","last_change":"2024-09-15T21:16:59.458213+0000","last_active":"2024-09-15T21:16:59.458409+0000","last_peered":"2024-09-15T21:16:59.458409+0000","last_clean":"2024-09-15T21:16:59.458409+0000","last_became_active":"2024-09-15T21:16:59.453118+0000","last_became_peered":"2024-09-15T21:16:59.453118+0000","last_unstale":"2024-09-15T21:16:59.458409+0000","last_undegraded":"2024-09-15T21:16:59.458409+0000","last_fullsized":"2024-09-15T21:16:59.458409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:03.427621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446971+0000","last_change":"2024-09-15T21:16:59.446702+0000","last_active":"2024-09-15T21:16:59.446971+0000","last_peered":"2024-09-15T21:16:59.446971+0000","last_clean":"2024-09-15T21:16:59.446971+0000","last_became_active":"2024-09-15T21:16:59.446093+0000","last_became_peered":"2024-09-15T21:16:59.446093+0000","last_unstale":"2024-09-15T21:16:59.446971+0000","last_undegraded":"2024-09-15T21:16:59.446971+0000","last_fullsized":"2024-09-15T21:16:59.446971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:13.220833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440780+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:59.440780+0000","last_peered":"2024-09-15T21:16:59.440780+0000","last_clean":"2024-09-15T21:16:59.440780+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:59.440780+0000","last_undegraded":"2024-09-15T21:16:59.440780+0000","last_fullsized":"2024-09-15T21:16:59.440780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439273+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:59.439273+0000","last_peered":"2024-09-15T21:16:59.439273+0000","last_clean":"2024-09-15T21:16:59.439273+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:59.439273+0000","last_undegraded":"2024-09-15T21:16:59.439273+0000","last_fullsized":"2024-09-15T21:16:59.439273+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440552+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:59.440552+0000","last_peered":"2024-09-15T21:16:59.440552+0000","last_clean":"2024-09-15T21:16:59.440552+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:59.440552+0000","last_undegraded":"2024-09-15T21:16:59.440552+0000","last_fullsized":"2024-09-15T21:16:59.440552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446013+0000","last_change":"2024-09-15T21:16:59.445756+0000","last_active":"2024-09-15T21:16:59.446013+0000","last_peered":"2024-09-15T21:16:59.446013+0000","last_clean":"2024-09-15T21:16:59.446013+0000","last_became_active":"2024-09-15T21:16:59.444954+0000","last_became_peered":"2024-09-15T21:16:59.444954+0000","last_unstale":"2024-09-15T21:16:59.446013+0000","last_undegraded":"2024-09-15T21:16:59.446013+0000","last_fullsized":"2024-09-15T21:16:59.446013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:26:16.369751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439912+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:59.439912+0000","last_peered":"2024-09-15T21:16:59.439912+0000","last_clean":"2024-09-15T21:16:59.439912+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:59.439912+0000","last_undegraded":"2024-09-15T21:16:59.439912+0000","last_fullsized":"2024-09-15T21:16:59.439912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440743+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:59.440743+0000","last_peered":"2024-09-15T21:16:59.440743+0000","last_clean":"2024-09-15T21:16:59.440743+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:59.440743+0000","last_undegraded":"2024-09-15T21:16:59.440743+0000","last_fullsized":"2024-09-15T21:16:59.440743+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.453189+0000","last_change":"2024-09-15T21:16:59.452491+0000","last_active":"2024-09-15T21:16:59.453189+0000","last_peered":"2024-09-15T21:16:59.453189+0000","last_clean":"2024-09-15T21:16:59.453189+0000","last_became_active":"2024-09-15T21:16:59.451619+0000","last_became_peered":"2024-09-15T21:16:59.451619+0000","last_unstale":"2024-09-15T21:16:59.453189+0000","last_undegraded":"2024-09-15T21:16:59.453189+0000","last_fullsized":"2024-09-15T21:16:59.453189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:31.903673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438957+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:59.438957+0000","last_peered":"2024-09-15T21:16:59.438957+0000","last_clean":"2024-09-15T21:16:59.438957+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:59.438957+0000","last_undegraded":"2024-09-15T21:16:59.438957+0000","last_fullsized":"2024-09-15T21:16:59.438957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:09.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:09.652 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:09.652 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:09.670 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-15T21:17:08.278520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54452,"kb_used_data":384,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689228,"statfs":{"total":193273528320,"available":193217769472,"internally_reserved":0,"allocated":393216,"data_stored":102990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3186,"internal_metadata":55309198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.851998"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.435209+0000","last_change":"2024-09-15T21:16:55.408383+0000","last_active":"2024-09-15T21:16:59.435209+0000","last_peered":"2024-09-15T21:16:59.435209+0000","last_clean":"2024-09-15T21:16:59.435209+0000","last_became_active":"2024-09-15T21:16:55.407657+0000","last_became_peered":"2024-09-15T21:16:55.407657+0000","last_unstale":"2024-09-15T21:16:59.435209+0000","last_undegraded":"2024-09-15T21:16:59.435209+0000","last_fullsized":"2024-09-15T21:16:59.435209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:53.537959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.447520+0000","last_change":"2024-09-15T21:16:59.445579+0000","last_active":"2024-09-15T21:16:59.447520+0000","last_peered":"2024-09-15T21:16:59.447520+0000","last_clean":"2024-09-15T21:16:59.447520+0000","last_became_active":"2024-09-15T21:16:59.444712+0000","last_became_peered":"2024-09-15T21:16:59.444712+0000","last_unstale":"2024-09-15T21:16:59.447520+0000","last_undegraded":"2024-09-15T21:16:59.447520+0000","last_fullsized":"2024-09-15T21:16:59.447520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:08:09.647705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.452843+0000","last_change":"2024-09-15T21:16:59.452558+0000","last_active":"2024-09-15T21:16:59.452843+0000","last_peered":"2024-09-15T21:16:59.452843+0000","last_clean":"2024-09-15T21:16:59.452843+0000","last_became_active":"2024-09-15T21:16:59.451802+0000","last_became_peered":"2024-09-15T21:16:59.451802+0000","last_unstale":"2024-09-15T21:16:59.452843+0000","last_undegraded":"2024-09-15T21:16:59.452843+0000","last_fullsized":"2024-09-15T21:16:59.452843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:19.006881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445384+0000","last_change":"2024-09-15T21:16:59.445105+0000","last_active":"2024-09-15T21:16:59.445384+0000","last_peered":"2024-09-15T21:16:59.445384+0000","last_clean":"2024-09-15T21:16:59.445384+0000","last_became_active":"2024-09-15T21:16:59.443707+0000","last_became_peered":"2024-09-15T21:16:59.443707+0000","last_unstale":"2024-09-15T21:16:59.445384+0000","last_undegraded":"2024-09-15T21:16:59.445384+0000","last_fullsized":"2024-09-15T21:16:59.445384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:00.666408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445850+0000","last_change":"2024-09-15T21:16:59.445580+0000","last_active":"2024-09-15T21:16:59.445850+0000","last_peered":"2024-09-15T21:16:59.445850+0000","last_clean":"2024-09-15T21:16:59.445850+0000","last_became_active":"2024-09-15T21:16:59.444782+0000","last_became_peered":"2024-09-15T21:16:59.444782+0000","last_unstale":"2024-09-15T21:16:59.445850+0000","last_undegraded":"2024-09-15T21:16:59.445850+0000","last_fullsized":"2024-09-15T21:16:59.445850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:26.734939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438948+0000","last_change":"2024-09-15T21:16:55.406121+0000","last_active":"2024-09-15T21:16:59.438948+0000","last_peered":"2024-09-15T21:16:59.438948+0000","last_clean":"2024-09-15T21:16:59.438948+0000","last_became_active":"2024-09-15T21:16:55.405143+0000","last_became_peered":"2024-09-15T21:16:55.405143+0000","last_unstale":"2024-09-15T21:16:59.438948+0000","last_undegraded":"2024-09-15T21:16:59.438948+0000","last_fullsized":"2024-09-15T21:16:59.438948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:26:10.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438682+0000","last_change":"2024-09-15T21:16:55.402955+0000","last_active":"2024-09-15T21:16:59.438682+0000","last_peered":"2024-09-15T21:16:59.438682+0000","last_clean":"2024-09-15T21:16:59.438682+0000","last_became_active":"2024-09-15T21:16:55.402687+0000","last_became_peered":"2024-09-15T21:16:55.402687+0000","last_unstale":"2024-09-15T21:16:59.438682+0000","last_undegraded":"2024-09-15T21:16:59.438682+0000","last_fullsized":"2024-09-15T21:16:59.438682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:54.700988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.448604+0000","last_change":"2024-09-15T21:16:59.448341+0000","last_active":"2024-09-15T21:16:59.448604+0000","last_peered":"2024-09-15T21:16:59.448604+0000","last_clean":"2024-09-15T21:16:59.448604+0000","last_became_active":"2024-09-15T21:16:59.447613+0000","last_became_peered":"2024-09-15T21:16:59.447613+0000","last_unstale":"2024-09-15T21:16:59.448604+0000","last_undegraded":"2024-09-15T21:16:59.448604+0000","last_fullsized":"2024-09-15T21:16:59.448604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:23.797403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438635+0000","last_change":"2024-09-15T21:16:46.354325+0000","last_active":"2024-09-15T21:16:59.438635+0000","last_peered":"2024-09-15T21:16:59.438635+0000","last_clean":"2024-09-15T21:16:59.438635+0000","last_became_active":"2024-09-15T21:16:44.354164+0000","last_became_peered":"2024-09-15T21:16:44.354164+0000","last_unstale":"2024-09-15T21:16:59.438635+0000","last_undegraded":"2024-09-15T21:16:59.438635+0000","last_fullsized":"2024-09-15T21:16:59.438635+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:05.909685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.451171+0000","last_change":"2024-09-15T21:16:59.450931+0000","last_active":"2024-09-15T21:16:59.451171+0000","last_peered":"2024-09-15T21:16:59.451171+0000","last_clean":"2024-09-15T21:16:59.451171+0000","last_became_active":"2024-09-15T21:16:59.450116+0000","last_became_peered":"2024-09-15T21:16:59.450116+0000","last_unstale":"2024-09-15T21:16:59.451171+0000","last_undegraded":"2024-09-15T21:16:59.451171+0000","last_fullsized":"2024-09-15T21:16:59.451171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:35.260779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.445580+0000","last_change":"2024-09-15T21:16:59.445350+0000","last_active":"2024-09-15T21:16:59.445580+0000","last_peered":"2024-09-15T21:16:59.445580+0000","last_clean":"2024-09-15T21:16:59.445580+0000","last_became_active":"2024-09-15T21:16:59.443592+0000","last_became_peered":"2024-09-15T21:16:59.443592+0000","last_unstale":"2024-09-15T21:16:59.445580+0000","last_undegraded":"2024-09-15T21:16:59.445580+0000","last_fullsized":"2024-09-15T21:16:59.445580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.251696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.434158+0000","last_change":"2024-09-15T21:16:55.408191+0000","last_active":"2024-09-15T21:16:59.434158+0000","last_peered":"2024-09-15T21:16:59.434158+0000","last_clean":"2024-09-15T21:16:59.434158+0000","last_became_active":"2024-09-15T21:16:55.407893+0000","last_became_peered":"2024-09-15T21:16:55.407893+0000","last_unstale":"2024-09-15T21:16:59.434158+0000","last_undegraded":"2024-09-15T21:16:59.434158+0000","last_fullsized":"2024-09-15T21:16:59.434158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:54.226322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":36,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.434425+0000","last_change":"2024-09-15T21:16:46.353429+0000","last_active":"2024-09-15T21:16:59.434425+0000","last_peered":"2024-09-15T21:16:59.434425+0000","last_clean":"2024-09-15T21:16:59.434425+0000","last_became_active":"2024-09-15T21:16:44.351819+0000","last_became_peered":"2024-09-15T21:16:44.351819+0000","last_unstale":"2024-09-15T21:16:59.434425+0000","last_undegraded":"2024-09-15T21:16:59.434425+0000","last_fullsized":"2024-09-15T21:16:59.434425+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:36.942346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.449422+0000","last_change":"2024-09-15T21:16:59.449193+0000","last_active":"2024-09-15T21:16:59.449422+0000","last_peered":"2024-09-15T21:16:59.449422+0000","last_clean":"2024-09-15T21:16:59.449422+0000","last_became_active":"2024-09-15T21:16:59.448448+0000","last_became_peered":"2024-09-15T21:16:59.448448+0000","last_unstale":"2024-09-15T21:16:59.449422+0000","last_undegraded":"2024-09-15T21:16:59.449422+0000","last_fullsized":"2024-09-15T21:16:59.449422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:56.837931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":44,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.458409+0000","last_change":"2024-09-15T21:16:59.458213+0000","last_active":"2024-09-15T21:16:59.458409+0000","last_peered":"2024-09-15T21:16:59.458409+0000","last_clean":"2024-09-15T21:16:59.458409+0000","last_became_active":"2024-09-15T21:16:59.453118+0000","last_became_peered":"2024-09-15T21:16:59.453118+0000","last_unstale":"2024-09-15T21:16:59.458409+0000","last_undegraded":"2024-09-15T21:16:59.458409+0000","last_fullsized":"2024-09-15T21:16:59.458409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:03.427621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446971+0000","last_change":"2024-09-15T21:16:59.446702+0000","last_active":"2024-09-15T21:16:59.446971+0000","last_peered":"2024-09-15T21:16:59.446971+0000","last_clean":"2024-09-15T21:16:59.446971+0000","last_became_active":"2024-09-15T21:16:59.446093+0000","last_became_peered":"2024-09-15T21:16:59.446093+0000","last_unstale":"2024-09-15T21:16:59.446971+0000","last_undegraded":"2024-09-15T21:16:59.446971+0000","last_fullsized":"2024-09-15T21:16:59.446971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:13.220833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440780+0000","last_change":"2024-09-15T21:16:55.405617+0000","last_active":"2024-09-15T21:16:59.440780+0000","last_peered":"2024-09-15T21:16:59.440780+0000","last_clean":"2024-09-15T21:16:59.440780+0000","last_became_active":"2024-09-15T21:16:55.404377+0000","last_became_peered":"2024-09-15T21:16:55.404377+0000","last_unstale":"2024-09-15T21:16:59.440780+0000","last_undegraded":"2024-09-15T21:16:59.440780+0000","last_fullsized":"2024-09-15T21:16:59.440780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:38.316650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439273+0000","last_change":"2024-09-15T21:16:46.356388+0000","last_active":"2024-09-15T21:16:59.439273+0000","last_peered":"2024-09-15T21:16:59.439273+0000","last_clean":"2024-09-15T21:16:59.439273+0000","last_became_active":"2024-09-15T21:16:44.351011+0000","last_became_peered":"2024-09-15T21:16:44.351011+0000","last_unstale":"2024-09-15T21:16:59.439273+0000","last_undegraded":"2024-09-15T21:16:59.439273+0000","last_fullsized":"2024-09-15T21:16:59.439273+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:11.146117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440552+0000","last_change":"2024-09-15T21:16:55.404635+0000","last_active":"2024-09-15T21:16:59.440552+0000","last_peered":"2024-09-15T21:16:59.440552+0000","last_clean":"2024-09-15T21:16:59.440552+0000","last_became_active":"2024-09-15T21:16:55.403761+0000","last_became_peered":"2024-09-15T21:16:55.403761+0000","last_unstale":"2024-09-15T21:16:59.440552+0000","last_undegraded":"2024-09-15T21:16:59.440552+0000","last_fullsized":"2024-09-15T21:16:59.440552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:39:05.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.446013+0000","last_change":"2024-09-15T21:16:59.445756+0000","last_active":"2024-09-15T21:16:59.446013+0000","last_peered":"2024-09-15T21:16:59.446013+0000","last_clean":"2024-09-15T21:16:59.446013+0000","last_became_active":"2024-09-15T21:16:59.444954+0000","last_became_peered":"2024-09-15T21:16:59.444954+0000","last_unstale":"2024-09-15T21:16:59.446013+0000","last_undegraded":"2024-09-15T21:16:59.446013+0000","last_fullsized":"2024-09-15T21:16:59.446013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:26:16.369751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.439912+0000","last_change":"2024-09-15T21:16:55.405650+0000","last_active":"2024-09-15T21:16:59.439912+0000","last_peered":"2024-09-15T21:16:59.439912+0000","last_clean":"2024-09-15T21:16:59.439912+0000","last_became_active":"2024-09-15T21:16:55.405257+0000","last_became_peered":"2024-09-15T21:16:55.405257+0000","last_unstale":"2024-09-15T21:16:59.439912+0000","last_undegraded":"2024-09-15T21:16:59.439912+0000","last_fullsized":"2024-09-15T21:16:59.439912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:28.678243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.440743+0000","last_change":"2024-09-15T21:16:46.355661+0000","last_active":"2024-09-15T21:16:59.440743+0000","last_peered":"2024-09-15T21:16:59.440743+0000","last_clean":"2024-09-15T21:16:59.440743+0000","last_became_active":"2024-09-15T21:16:44.351150+0000","last_became_peered":"2024-09-15T21:16:44.351150+0000","last_unstale":"2024-09-15T21:16:59.440743+0000","last_undegraded":"2024-09-15T21:16:59.440743+0000","last_fullsized":"2024-09-15T21:16:59.440743+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:57.062660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.453189+0000","last_change":"2024-09-15T21:16:59.452491+0000","last_active":"2024-09-15T21:16:59.453189+0000","last_peered":"2024-09-15T21:16:59.453189+0000","last_clean":"2024-09-15T21:16:59.453189+0000","last_became_active":"2024-09-15T21:16:59.451619+0000","last_became_peered":"2024-09-15T21:16:59.451619+0000","last_unstale":"2024-09-15T21:16:59.453189+0000","last_undegraded":"2024-09-15T21:16:59.453189+0000","last_fullsized":"2024-09-15T21:16:59.453189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:31.903673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:16:59.438957+0000","last_change":"2024-09-15T21:16:55.406086+0000","last_active":"2024-09-15T21:16:59.438957+0000","last_peered":"2024-09-15T21:16:59.438957+0000","last_clean":"2024-09-15T21:16:59.438957+0000","last_became_active":"2024-09-15T21:16:55.404837+0000","last_became_peered":"2024-09-15T21:16:55.404837+0000","last_unstale":"2024-09-15T21:16:59.438957+0000","last_undegraded":"2024-09-15T21:16:59.438957+0000","last_fullsized":"2024-09-15T21:16:59.438957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:03.287833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672971,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738379,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":51495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:09.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-15T21:17:09.671 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-15T21:17:09.671 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-15T21:17:09.717 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-15T21:17:09.718 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-15T21:17:09.758 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-15T21:17:09.758 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-15T21:17:09.840 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-15T21:17:09.840 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-15T21:17:09.926 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-15T21:17:09.926 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-15T21:17:10.015 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-15T21:17:10.015 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-15T21:17:10.102 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-15T21:17:10.102 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-15T21:17:10.188 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-15T21:17:10.188 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-15T21:17:10.277 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-15T21:17:10.277 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-15T21:17:10.317 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-15T21:17:10.317 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-15T21:17:10.407 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-15T21:17:10.407 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-15T21:17:10.448 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-15T21:17:10.448 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-15T21:17:10.532 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-15T21:17:10.532 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-15T21:17:10.621 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-15T21:17:10.621 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-15T21:17:10.707 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-15T21:17:10.707 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-15T21:17:10.794 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-15T21:17:10.794 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-15T21:17:10.835 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-15T21:17:10.836 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-15T21:17:10.918 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-15T21:17:10.918 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-15T21:17:11.010 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-15T21:17:11.010 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-15T21:17:11.095 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-15T21:17:11.095 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-15T21:17:11.179 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-15T21:17:11.179 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-15T21:17:11.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:17:11.450+0000 7f54fb068640 0 --2- 172.21.15.114:0/2182875240 >> [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] conn(0x7f54e8001dd0 0x7f54e8004220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:17:11.468 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-15T21:17:11.468 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-15T21:17:11.511 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-15T21:17:11.511 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-15T21:17:11.600 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-15T21:17:11.601 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-15T21:17:11.640 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-15T21:17:11.640 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-15T21:17:11.725 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-15T21:17:11.726 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-15T21:17:11.809 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-15T21:17:11.809 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-15T21:17:11.892 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-15T21:17:12.084 INFO:teuthology.orchestra.run.smithi114.stdout:{} 2024-09-15T21:17:12.084 INFO:teuthology.orchestra.run.smithi114.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-15T21:17:12.101 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:17:12.101 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:17:12.191 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:17:12.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-15T21:17:12.509 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:12.506+0000 7fefb254a640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2024-09-15T21:17:12.509 INFO:teuthology.orchestra.run.smithi114.stderr:osd.0 is already down. 2024-09-15T21:17:12.527 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-15T21:17:12.527 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-15T21:17:13.506 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-15T21:17:13.507 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-15T21:17:13.555 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-15T21:17:13.555 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-15T21:17:13.597 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-15T21:17:13.597 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-15T21:17:13.680 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-15T21:17:13.680 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-15T21:17:13.767 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-15T21:17:13.767 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-15T21:17:13.851 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-15T21:17:13.851 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-15T21:17:13.930 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-15T21:17:13.930 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-15T21:17:14.014 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-15T21:17:14.014 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-15T21:17:14.101 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-15T21:17:14.101 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-15T21:17:14.189 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-15T21:17:14.189 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-15T21:17:14.227 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-15T21:17:14.228 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-15T21:17:14.308 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-15T21:17:14.309 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-15T21:17:14.388 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-15T21:17:14.388 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-15T21:17:14.473 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-15T21:17:14.473 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-15T21:17:14.556 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-15T21:17:14.556 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-15T21:17:14.640 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-15T21:17:14.640 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-15T21:17:14.723 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-15T21:17:14.723 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-15T21:17:14.802 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-15T21:17:14.802 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-15T21:17:14.891 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-15T21:17:14.891 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-15T21:17:14.978 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-15T21:17:14.978 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-15T21:17:15.066 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-15T21:17:15.066 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-15T21:17:15.105 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-15T21:17:15.105 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-15T21:17:15.188 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-15T21:17:15.188 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-15T21:17:15.272 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-15T21:17:15.272 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-15T21:17:15.360 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-15T21:17:15.360 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-15T21:17:15.447 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-15T21:17:15.448 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-15T21:17:15.531 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-15T21:17:15.531 INFO:tasks.rep_lost_unfound_delete:() 2024-09-15T21:17:15.531 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:17:15.532 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-15T21:17:15.573 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:17:15.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-15T21:17:15.690 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-15T21:17:15.691 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-15T21:17:15.691 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-15T21:17:15.691 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:17:15.692 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:17:15.692 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T21:17:16.678 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:16.674+0000 7f8e3c8ac7c0 -1 Falling back to public interface 2024-09-15T21:17:20.615 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:20.610+0000 7f8e3c8ac7c0 -1 osd.0 22 log_to_monitors true 2024-09-15T21:17:20.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs files list 2024-09-15T21:17:20.817 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs stats 2024-09-15T21:17:20.818 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator dump block 2024-09-15T21:17:20.818 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation block 2024-09-15T21:17:20.818 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:17:20.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:17:20.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T21:17:21.565 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:21.562+0000 7f8e30687640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:25.822 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:17:25.956 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:17:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-15T21:17:25.957 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-15T21:17:25.957 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:17:25.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-15T21:17:26.284 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:26.278+0000 7fefb254a640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-15T21:17:26.284 INFO:teuthology.orchestra.run.smithi114.stderr:osd.0 is already in. 2024-09-15T21:17:26.302 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-15T21:17:26.303 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":27,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:17:23.571784+0000","last_up_change":"2024-09-15T21:17:22.551857+0000","last_in_change":"2024-09-15T21:16:58.414173+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-15T21:16:54.321157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":26,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":490656443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":490656443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":490656443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":490656443}]},"public_addr":"172.21.15.114:6804/490656443","cluster_addr":"172.21.15.114:6805/490656443","heartbeat_back_addr":"172.21.15.114:6807/490656443","heartbeat_front_addr":"172.21.15.114:6806/490656443","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2337399995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":2337399995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":2337399995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2337399995}]},"public_addr":"172.21.15.114:6804/2337399995","cluster_addr":"172.21.15.114:6805/2337399995","heartbeat_back_addr":"172.21.15.114:6807/2337399995","heartbeat_front_addr":"172.21.15.114:6806/2337399995","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:26.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-15T21:17:26.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:17:26.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:17:26.775 INFO:teuthology.orchestra.run.smithi114.stdout:111669149699 2024-09-15T21:17:26.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:26.781 INFO:teuthology.orchestra.run.smithi114.stdout:34359738384 2024-09-15T21:17:26.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:27.058 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-15T21:17:27.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 1 for osd.0 2024-09-15T21:17:27.110 INFO:teuthology.orchestra.run.smithi114.stdout:34359738382 2024-09-15T21:17:27.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738384 got 34359738382 for osd.1 2024-09-15T21:17:28.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:28.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:28.400 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-15T21:17:28.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 1 for osd.0 2024-09-15T21:17:28.418 INFO:teuthology.orchestra.run.smithi114.stdout:34359738382 2024-09-15T21:17:28.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738384 got 34359738382 for osd.1 2024-09-15T21:17:29.418 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:29.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:17:29.737 INFO:teuthology.orchestra.run.smithi114.stdout:111669149699 2024-09-15T21:17:29.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149699 got 111669149699 for osd.0 2024-09-15T21:17:29.758 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:29.778 INFO:teuthology.orchestra.run.smithi114.stdout:34359738384 2024-09-15T21:17:29.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738384 got 34359738384 for osd.1 2024-09-15T21:17:29.797 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:29.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-15T21:17:29.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:30.134 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:30.135 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:30.151 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-09-15T21:17:28.283728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55112,"kb_used_data":608,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688568,"statfs":{"total":193273528320,"available":193217093632,"internally_reserved":0,"allocated":622592,"data_stored":156170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6026,"internal_metadata":55765110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.738365"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.618666+0000","last_change":"2024-09-15T21:17:23.618530+0000","last_active":"2024-09-15T21:17:23.618666+0000","last_peered":"2024-09-15T21:17:23.618666+0000","last_clean":"2024-09-15T21:17:23.618666+0000","last_became_active":"2024-09-15T21:17:23.618012+0000","last_became_peered":"2024-09-15T21:17:23.618012+0000","last_unstale":"2024-09-15T21:17:23.618666+0000","last_undegraded":"2024-09-15T21:17:23.618666+0000","last_fullsized":"2024-09-15T21:17:23.618666+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:40.896195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":25,"reported_epoch":27,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:17:23.617419+0000","last_change":"2024-09-15T21:17:23.622257+0000","last_active":"2024-09-15T21:17:23.617419+0000","last_peered":"2024-09-15T21:17:23.617419+0000","last_clean":"2024-09-15T21:17:10.403563+0000","last_became_active":"2024-09-15T21:17:23.616891+0000","last_became_peered":"2024-09-15T21:17:23.616891+0000","last_unstale":"2024-09-15T21:17:23.617419+0000","last_undegraded":"2024-09-15T21:17:23.598865+0000","last_fullsized":"2024-09-15T21:17:23.617419+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":44,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.610218+0000","last_change":"2024-09-15T21:17:23.610748+0000","last_active":"2024-09-15T21:17:23.610218+0000","last_peered":"2024-09-15T21:17:23.610218+0000","last_clean":"2024-09-15T21:17:11.463614+0000","last_became_active":"2024-09-15T21:17:23.610008+0000","last_became_peered":"2024-09-15T21:17:23.610008+0000","last_unstale":"2024-09-15T21:17:23.610218+0000","last_undegraded":"2024-09-15T21:17:23.592673+0000","last_fullsized":"2024-09-15T21:17:23.610218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.621027+0000","last_change":"2024-09-15T21:17:23.620832+0000","last_active":"2024-09-15T21:17:23.621027+0000","last_peered":"2024-09-15T21:17:23.621027+0000","last_clean":"2024-09-15T21:17:23.621027+0000","last_became_active":"2024-09-15T21:17:23.620094+0000","last_became_peered":"2024-09-15T21:17:23.620094+0000","last_unstale":"2024-09-15T21:17:23.621027+0000","last_undegraded":"2024-09-15T21:17:23.621027+0000","last_fullsized":"2024-09-15T21:17:23.621027+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:08.291960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":36,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.617801+0000","last_change":"2024-09-15T21:17:23.618285+0000","last_active":"2024-09-15T21:17:23.617801+0000","last_peered":"2024-09-15T21:17:23.617801+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:23.617533+0000","last_became_peered":"2024-09-15T21:17:23.617533+0000","last_unstale":"2024-09-15T21:17:23.617801+0000","last_undegraded":"2024-09-15T21:17:23.600399+0000","last_fullsized":"2024-09-15T21:17:23.617801+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":39,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.609783+0000","last_change":"2024-09-15T21:17:23.610169+0000","last_active":"2024-09-15T21:17:23.609783+0000","last_peered":"2024-09-15T21:17:23.609783+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:23.609530+0000","last_became_peered":"2024-09-15T21:17:23.609530+0000","last_unstale":"2024-09-15T21:17:23.609783+0000","last_undegraded":"2024-09-15T21:17:23.591916+0000","last_fullsized":"2024-09-15T21:17:23.609783+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":40,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.610755+0000","last_change":"2024-09-15T21:17:23.614478+0000","last_active":"2024-09-15T21:17:23.610755+0000","last_peered":"2024-09-15T21:17:23.610755+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:23.610438+0000","last_became_peered":"2024-09-15T21:17:23.610438+0000","last_unstale":"2024-09-15T21:17:23.610755+0000","last_undegraded":"2024-09-15T21:17:23.592319+0000","last_fullsized":"2024-09-15T21:17:23.610755+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":33,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.619155+0000","last_change":"2024-09-15T21:17:23.620502+0000","last_active":"2024-09-15T21:17:23.619155+0000","last_peered":"2024-09-15T21:17:23.619155+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:23.618996+0000","last_became_peered":"2024-09-15T21:17:23.618996+0000","last_unstale":"2024-09-15T21:17:23.619155+0000","last_undegraded":"2024-09-15T21:17:23.603955+0000","last_fullsized":"2024-09-15T21:17:23.619155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.613120+0000","last_change":"2024-09-15T21:17:23.612902+0000","last_active":"2024-09-15T21:17:23.613120+0000","last_peered":"2024-09-15T21:17:23.613120+0000","last_clean":"2024-09-15T21:17:23.613120+0000","last_became_active":"2024-09-15T21:17:23.610470+0000","last_became_peered":"2024-09-15T21:17:23.610470+0000","last_unstale":"2024-09-15T21:17:23.613120+0000","last_undegraded":"2024-09-15T21:17:23.613120+0000","last_fullsized":"2024-09-15T21:17:23.613120+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:06.938585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":46,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.605356+0000","last_change":"2024-09-15T21:17:23.605824+0000","last_active":"2024-09-15T21:17:23.605356+0000","last_peered":"2024-09-15T21:17:23.605356+0000","last_clean":"2024-09-15T21:17:11.507438+0000","last_became_active":"2024-09-15T21:17:23.605110+0000","last_became_peered":"2024-09-15T21:17:23.605110+0000","last_unstale":"2024-09-15T21:17:23.605356+0000","last_undegraded":"2024-09-15T21:17:23.594192+0000","last_fullsized":"2024-09-15T21:17:23.605356+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.618651+0000","last_change":"2024-09-15T21:17:23.618400+0000","last_active":"2024-09-15T21:17:23.618651+0000","last_peered":"2024-09-15T21:17:23.618651+0000","last_clean":"2024-09-15T21:17:23.618651+0000","last_became_active":"2024-09-15T21:17:23.617593+0000","last_became_peered":"2024-09-15T21:17:23.617593+0000","last_unstale":"2024-09-15T21:17:23.618651+0000","last_undegraded":"2024-09-15T21:17:23.618651+0000","last_fullsized":"2024-09-15T21:17:23.618651+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:17.722696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":32,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.617529+0000","last_change":"2024-09-15T21:17:23.617831+0000","last_active":"2024-09-15T21:17:23.617529+0000","last_peered":"2024-09-15T21:17:23.617529+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:23.617365+0000","last_became_peered":"2024-09-15T21:17:23.617365+0000","last_unstale":"2024-09-15T21:17:23.617529+0000","last_undegraded":"2024-09-15T21:17:23.597841+0000","last_fullsized":"2024-09-15T21:17:23.617529+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.618382+0000","last_change":"2024-09-15T21:17:23.618191+0000","last_active":"2024-09-15T21:17:23.618382+0000","last_peered":"2024-09-15T21:17:23.618382+0000","last_clean":"2024-09-15T21:17:23.618382+0000","last_became_active":"2024-09-15T21:17:23.617579+0000","last_became_peered":"2024-09-15T21:17:23.617579+0000","last_unstale":"2024-09-15T21:17:23.618382+0000","last_undegraded":"2024-09-15T21:17:23.618382+0000","last_fullsized":"2024-09-15T21:17:23.618382+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:57.686298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":33,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.613409+0000","last_change":"2024-09-15T21:17:23.613892+0000","last_active":"2024-09-15T21:17:23.613409+0000","last_peered":"2024-09-15T21:17:23.613409+0000","last_clean":"2024-09-15T21:17:10.184528+0000","last_became_active":"2024-09-15T21:17:23.613171+0000","last_became_peered":"2024-09-15T21:17:23.613171+0000","last_unstale":"2024-09-15T21:17:23.613409+0000","last_undegraded":"2024-09-15T21:17:23.591861+0000","last_fullsized":"2024-09-15T21:17:23.613409+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.613137+0000","last_change":"2024-09-15T21:17:23.612842+0000","last_active":"2024-09-15T21:17:23.613137+0000","last_peered":"2024-09-15T21:17:23.613137+0000","last_clean":"2024-09-15T21:17:23.613137+0000","last_became_active":"2024-09-15T21:17:23.609386+0000","last_became_peered":"2024-09-15T21:17:23.609386+0000","last_unstale":"2024-09-15T21:17:23.613137+0000","last_undegraded":"2024-09-15T21:17:23.613137+0000","last_fullsized":"2024-09-15T21:17:23.613137+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:37.123234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":34,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.609606+0000","last_change":"2024-09-15T21:17:23.610490+0000","last_active":"2024-09-15T21:17:23.609606+0000","last_peered":"2024-09-15T21:17:23.609606+0000","last_clean":"2024-09-15T21:17:10.790830+0000","last_became_active":"2024-09-15T21:17:23.609353+0000","last_became_peered":"2024-09-15T21:17:23.609353+0000","last_unstale":"2024-09-15T21:17:23.609606+0000","last_undegraded":"2024-09-15T21:17:23.591711+0000","last_fullsized":"2024-09-15T21:17:23.609606+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":36,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.610271+0000","last_change":"2024-09-15T21:17:23.610806+0000","last_active":"2024-09-15T21:17:23.610271+0000","last_peered":"2024-09-15T21:17:23.610271+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:23.610042+0000","last_became_peered":"2024-09-15T21:17:23.610042+0000","last_unstale":"2024-09-15T21:17:23.610271+0000","last_undegraded":"2024-09-15T21:17:23.596171+0000","last_fullsized":"2024-09-15T21:17:23.610271+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.605830+0000","last_change":"2024-09-15T21:17:23.605637+0000","last_active":"2024-09-15T21:17:23.605830+0000","last_peered":"2024-09-15T21:17:23.605830+0000","last_clean":"2024-09-15T21:17:23.605830+0000","last_became_active":"2024-09-15T21:17:23.604857+0000","last_became_peered":"2024-09-15T21:17:23.604857+0000","last_unstale":"2024-09-15T21:17:23.605830+0000","last_undegraded":"2024-09-15T21:17:23.605830+0000","last_fullsized":"2024-09-15T21:17:23.605830+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:50.539413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":42,"reported_epoch":27,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:17:23.604121+0000","last_change":"2024-09-15T21:17:23.619613+0000","last_active":"2024-09-15T21:17:23.604121+0000","last_peered":"2024-09-15T21:17:23.604121+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:23.603466+0000","last_became_peered":"2024-09-15T21:17:23.603466+0000","last_unstale":"2024-09-15T21:17:23.604121+0000","last_undegraded":"2024-09-15T21:17:23.595987+0000","last_fullsized":"2024-09-15T21:17:23.604121+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.620531+0000","last_change":"2024-09-15T21:17:23.619997+0000","last_active":"2024-09-15T21:17:23.620531+0000","last_peered":"2024-09-15T21:17:23.620531+0000","last_clean":"2024-09-15T21:17:23.620531+0000","last_became_active":"2024-09-15T21:17:23.618619+0000","last_became_peered":"2024-09-15T21:17:23.618619+0000","last_unstale":"2024-09-15T21:17:23.620531+0000","last_undegraded":"2024-09-15T21:17:23.620531+0000","last_fullsized":"2024-09-15T21:17:23.620531+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:13:51.476741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":37,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.612367+0000","last_change":"2024-09-15T21:17:23.612750+0000","last_active":"2024-09-15T21:17:23.612367+0000","last_peered":"2024-09-15T21:17:23.612367+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:23.612171+0000","last_became_peered":"2024-09-15T21:17:23.612171+0000","last_unstale":"2024-09-15T21:17:23.612367+0000","last_undegraded":"2024-09-15T21:17:23.593425+0000","last_fullsized":"2024-09-15T21:17:23.612367+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.610468+0000","last_change":"2024-09-15T21:17:23.610173+0000","last_active":"2024-09-15T21:17:23.610468+0000","last_peered":"2024-09-15T21:17:23.610468+0000","last_clean":"2024-09-15T21:17:23.610468+0000","last_became_active":"2024-09-15T21:17:23.609373+0000","last_became_peered":"2024-09-15T21:17:23.609373+0000","last_unstale":"2024-09-15T21:17:23.610468+0000","last_undegraded":"2024-09-15T21:17:23.610468+0000","last_fullsized":"2024-09-15T21:17:23.610468+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:46.859588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-15T21:17:23.616899+0000","last_change":"2024-09-15T21:17:23.615735+0000","last_active":"2024-09-15T21:17:23.616899+0000","last_peered":"2024-09-15T21:17:23.616899+0000","last_clean":"2024-09-15T21:17:23.616899+0000","last_became_active":"2024-09-15T21:17:23.612282+0000","last_became_peered":"2024-09-15T21:17:23.612282+0000","last_unstale":"2024-09-15T21:17:23.616899+0000","last_undegraded":"2024-09-15T21:17:23.616899+0000","last_fullsized":"2024-09-15T21:17:23.616899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:58.021070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":45,"reported_epoch":27,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:23.612053+0000","last_change":"2024-09-15T21:17:23.614296+0000","last_active":"2024-09-15T21:17:23.612053+0000","last_peered":"2024-09-15T21:17:23.612053+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:23.610674+0000","last_became_peered":"2024-09-15T21:17:23.610674+0000","last_unstale":"2024-09-15T21:17:23.612053+0000","last_undegraded":"2024-09-15T21:17:23.592381+0000","last_fullsized":"2024-09-15T21:17:23.612053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149699,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":274432,"data_stored":72532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":28110513},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":348160,"data_stored":83638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:30.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:17:30.484 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:30.485 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":63,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":27,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435042,"num_in_osds":2,"osd_in_since":1726435018,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56434688,"bytes_avail":193217093632,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-15T21:16:26.968475+0000","services":{}},"progress_events":{}} 2024-09-15T21:17:30.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '84ba6ec4-065b-47f6-87f1-d15da93be3d3', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 63, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 27, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435042, 'num_in_osds': 2, 'osd_in_since': 1726435018, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56434688, 'bytes_avail': 193217093632, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:16:26:965950+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-15T21:16:26.968475+0000', 'services': {}}, 'progress_events': {}} 2024-09-15T21:17:30.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-15T21:17:30.502 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:17:30.502 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:17:30.570 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:17:30.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-15T21:17:30.880 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:30.874+0000 7fefb254a640 -1 mon.a@0(leader).osd e28 definitely_dead 0 2024-09-15T21:17:30.880 INFO:teuthology.orchestra.run.smithi114.stderr:osd.1 is already down. 2024-09-15T21:17:30.897 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-15T21:17:31.180 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:31.178+0000 7fefb254a640 -1 mon.a@0(leader).osd e28 definitely_dead 0 2024-09-15T21:17:31.766 INFO:teuthology.orchestra.run.smithi114.stderr:marked out osd.1. 2024-09-15T21:17:31.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-15T21:17:32.777 INFO:teuthology.orchestra.run.smithi114.stderr:marked osd lost in epoch 28 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1a515640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1ed1e640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1a515640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1ad16640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:17:32.783 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1b517640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e19d14640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e1cd1a640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e19513640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e19513640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:17:32.784 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:32.778+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:32.804 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:17:32.805 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:17:32.807 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:17:32.807 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:17:32.916 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:17:32.916 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-15T21:17:32.917 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:17:32.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:17:32.919 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T21:17:33.910 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:17:33.906+0000 7f96ca5f07c0 -1 Falling back to public interface 2024-09-15T21:17:37.815 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:17:37.810+0000 7f96ca5f07c0 -1 osd.2 19 log_to_monitors true 2024-09-15T21:17:37.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:17:38.044 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:17:38.044 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:17:38.044 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs files list 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs stats 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator dump block 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation block 2024-09-15T21:17:38.045 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:17:38.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:17:38.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T21:17:38.365 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1ad16640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.366 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1a515640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.366 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:38.366 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1e51d640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.366 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1ad16640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:17:38.367 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e19d14640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.367 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e18d12640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:38.367 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e19513640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.367 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1d51b640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:17:38.370 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:38.370 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:17:38.370 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1bd18640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:38.370 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1c519640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:17:38.370 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:38.362+0000 7f8e1b517640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:38.376 INFO:tasks.ceph.osd.2.smithi114.stderr:2024-09-15T21:17:38.374+0000 7f96be3cb640 -1 osd.2 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:17:39.381 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1ed1e640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:17:39.381 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e19513640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1e51d640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1d51b640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1ad16640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e19d14640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1e51d640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.374+0000 7f8e1cd1a640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.378+0000 7f8e19d14640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.378+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.378+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.378+0000 7f8e1750f640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:39.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:39.378+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:40.383 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.378+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:40.383 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.378+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:40.385 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.382+0000 7f8e1b517640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:40.386 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.382+0000 7f8e1bd18640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:40.387 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.382+0000 7f8e1cd1a640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:40.392 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.386+0000 7f8e1a515640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:40.394 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:40.386+0000 7f8e1c519640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:41.377 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e18d12640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:41.377 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:41.378 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:41.379 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:41.380 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e1a515640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:41.381 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e1b517640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:41.382 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:41.374+0000 7f8e1c519640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:43.048 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-15T21:17:43.184 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:17:43.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-15T21:17:43.515 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:43.510+0000 7fefb254a640 -1 mon.a@0(leader).osd e34 definitely_dead 0 2024-09-15T21:17:44.415 INFO:teuthology.orchestra.run.smithi114.stderr:marked in osd.2. 2024-09-15T21:17:44.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-15T21:17:44.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:17:44.746 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:44.747 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":35,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:17:44.397344+0000","last_up_change":"2024-09-15T21:17:39.358083+0000","last_in_change":"2024-09-15T21:17:44.397344+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-15T21:16:54.321157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":33,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":490656443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":490656443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":490656443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":490656443}]},"public_addr":"172.21.15.114:6804/490656443","cluster_addr":"172.21.15.114:6805/490656443","heartbeat_back_addr":"172.21.15.114:6807/490656443","heartbeat_front_addr":"172.21.15.114:6806/490656443","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":26,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2054907799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":2054907799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":2054907799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2054907799}]},"public_addr":"172.21.15.114:6800/2054907799","cluster_addr":"172.21.15.114:6801/2054907799","heartbeat_back_addr":"172.21.15.114:6803/2054907799","heartbeat_front_addr":"172.21.15.114:6802/2054907799","state":["exists"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":19,"up_from":32,"up_thru":17,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":42043953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":42043953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":42043953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":42043953}]},"public_addr":"172.21.15.114:6800/42043953","cluster_addr":"172.21.15.114:6801/42043953","heartbeat_back_addr":"172.21.15.114:6803/42043953","heartbeat_front_addr":"172.21.15.114:6802/42043953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:17:44.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-15T21:17:44.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:17:44.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:17:44.925 INFO:teuthology.orchestra.run.smithi114.stdout:111669149703 2024-09-15T21:17:44.926 INFO:teuthology.orchestra.run.smithi114.stdout:137438953475 2024-09-15T21:17:44.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:17:44.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:45.220 INFO:teuthology.orchestra.run.smithi114.stdout:111669149702 2024-09-15T21:17:45.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149703 got 111669149702 for osd.0 2024-09-15T21:17:45.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:17:45.242+0000 7f9b5ad9d640 0 --2- 172.21.15.114:0/3471125950 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f9b5c12b220 0x7f9b5c12d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:17:45.427 INFO:teuthology.orchestra.run.smithi114.stdout:137438953474 2024-09-15T21:17:45.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953475 got 137438953474 for osd.2 2024-09-15T21:17:46.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:46.434 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e1ed1e640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:17:46.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:17:46.464 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e19d14640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:17:46.464 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e1dd1c640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:17:46.464 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e1e51d640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:17:46.464 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e17d10640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:17:46.464 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e18d12640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:17:46.465 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e18511640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:17:46.465 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e1750f640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:17:46.465 INFO:tasks.ceph.osd.0.smithi114.stderr:2024-09-15T21:17:46.429+0000 7f8e19513640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:17:46.554 INFO:teuthology.orchestra.run.smithi114.stdout:111669149703 2024-09-15T21:17:46.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149703 got 111669149703 for osd.0 2024-09-15T21:17:46.572 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:46.759 INFO:teuthology.orchestra.run.smithi114.stdout:137438953475 2024-09-15T21:17:46.778 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953475 got 137438953475 for osd.2 2024-09-15T21:17:46.778 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:46.778 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-15T21:17:46.778 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:47.067 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:47.068 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:47.084 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-15T21:17:46.428925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55636,"kb_used_data":620,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688044,"statfs":{"total":193273528320,"available":193216557056,"internally_reserved":0,"allocated":634880,"data_stored":188545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":56286558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.054935"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.379427+0000","last_change":"2024-09-15T21:17:44.423043+0000","last_active":"2024-09-15T21:17:44.423043+0000","last_peered":"2024-09-15T21:17:41.379427+0000","last_clean":"2024-09-15T21:17:23.622382+0000","last_became_active":"2024-09-15T21:17:31.774955+0000","last_became_peered":"2024-09-15T21:17:31.774955+0000","last_unstale":"2024-09-15T21:17:41.379427+0000","last_undegraded":"2024-09-15T21:17:41.379427+0000","last_fullsized":"2024-09-15T21:17:31.763822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":45,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.395515+0000","last_change":"2024-09-15T21:17:44.420859+0000","last_active":"2024-09-15T21:17:44.420859+0000","last_peered":"2024-09-15T21:17:41.395515+0000","last_clean":"2024-09-15T21:17:10.403563+0000","last_became_active":"2024-09-15T21:17:41.394768+0000","last_became_peered":"2024-09-15T21:17:41.394768+0000","last_unstale":"2024-09-15T21:17:41.395515+0000","last_undegraded":"2024-09-15T21:17:41.377959+0000","last_fullsized":"2024-09-15T21:17:41.395515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":60,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.397126+0000","last_change":"2024-09-15T21:17:44.420579+0000","last_active":"2024-09-15T21:17:44.420579+0000","last_peered":"2024-09-15T21:17:41.397126+0000","last_clean":"2024-09-15T21:17:11.463614+0000","last_became_active":"2024-09-15T21:17:41.395983+0000","last_became_peered":"2024-09-15T21:17:41.395983+0000","last_unstale":"2024-09-15T21:17:41.397126+0000","last_undegraded":"2024-09-15T21:17:41.379460+0000","last_fullsized":"2024-09-15T21:17:41.397126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.399937+0000","last_change":"2024-09-15T21:17:44.421273+0000","last_active":"2024-09-15T21:17:44.421273+0000","last_peered":"2024-09-15T21:17:41.399937+0000","last_clean":"2024-09-15T21:17:41.399937+0000","last_became_active":"2024-09-15T21:17:41.394779+0000","last_became_peered":"2024-09-15T21:17:41.394779+0000","last_unstale":"2024-09-15T21:17:41.399937+0000","last_undegraded":"2024-09-15T21:17:41.399937+0000","last_fullsized":"2024-09-15T21:17:41.399937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":57,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-15T21:17:44.423082+0000","last_change":"2024-09-15T21:17:44.423082+0000","last_active":"2024-09-15T21:17:44.423082+0000","last_peered":"2024-09-15T21:17:41.394332+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:41.393712+0000","last_became_peered":"2024-09-15T21:17:41.393712+0000","last_unstale":"2024-09-15T21:17:44.423082+0000","last_undegraded":"2024-09-15T21:17:44.423082+0000","last_fullsized":"2024-09-15T21:17:44.423082+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":52,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.377173+0000","last_change":"2024-09-15T21:17:44.422475+0000","last_active":"2024-09-15T21:17:44.422475+0000","last_peered":"2024-09-15T21:17:41.377173+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:31.766016+0000","last_became_peered":"2024-09-15T21:17:31.766016+0000","last_unstale":"2024-09-15T21:17:41.377173+0000","last_undegraded":"2024-09-15T21:17:31.760606+0000","last_fullsized":"2024-09-15T21:17:31.760268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":54,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.378819+0000","last_change":"2024-09-15T21:17:44.424893+0000","last_active":"2024-09-15T21:17:44.424893+0000","last_peered":"2024-09-15T21:17:41.378819+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:31.765953+0000","last_became_peered":"2024-09-15T21:17:31.765953+0000","last_unstale":"2024-09-15T21:17:41.378819+0000","last_undegraded":"2024-09-15T21:17:31.760479+0000","last_fullsized":"2024-09-15T21:17:31.760128+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":54,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-15T21:17:44.420540+0000","last_change":"2024-09-15T21:17:44.420540+0000","last_active":"2024-09-15T21:17:44.420540+0000","last_peered":"2024-09-15T21:17:41.399334+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:41.392575+0000","last_became_peered":"2024-09-15T21:17:41.392575+0000","last_unstale":"2024-09-15T21:17:44.420540+0000","last_undegraded":"2024-09-15T21:17:44.420540+0000","last_fullsized":"2024-09-15T21:17:44.420540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.376188+0000","last_change":"2024-09-15T21:17:44.421727+0000","last_active":"2024-09-15T21:17:44.421727+0000","last_peered":"2024-09-15T21:17:41.376188+0000","last_clean":"2024-09-15T21:16:59.438754+0000","last_became_active":"2024-09-15T21:17:31.766112+0000","last_became_peered":"2024-09-15T21:17:31.766112+0000","last_unstale":"2024-09-15T21:17:41.376188+0000","last_undegraded":"2024-09-15T21:17:41.376188+0000","last_fullsized":"2024-09-15T21:17:31.759220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":62,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.393512+0000","last_change":"2024-09-15T21:17:44.421304+0000","last_active":"2024-09-15T21:17:44.421304+0000","last_peered":"2024-09-15T21:17:41.393512+0000","last_clean":"2024-09-15T21:17:11.507438+0000","last_became_active":"2024-09-15T21:17:41.392788+0000","last_became_peered":"2024-09-15T21:17:41.392788+0000","last_unstale":"2024-09-15T21:17:41.393512+0000","last_undegraded":"2024-09-15T21:17:41.380222+0000","last_fullsized":"2024-09-15T21:17:41.393512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.397570+0000","last_change":"2024-09-15T21:17:44.421264+0000","last_active":"2024-09-15T21:17:44.421264+0000","last_peered":"2024-09-15T21:17:41.397570+0000","last_clean":"2024-09-15T21:17:41.397570+0000","last_became_active":"2024-09-15T21:17:41.392520+0000","last_became_peered":"2024-09-15T21:17:41.392520+0000","last_unstale":"2024-09-15T21:17:41.397570+0000","last_undegraded":"2024-09-15T21:17:41.397570+0000","last_fullsized":"2024-09-15T21:17:41.397570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":49,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.377716+0000","last_change":"2024-09-15T21:17:44.423756+0000","last_active":"2024-09-15T21:17:44.423756+0000","last_peered":"2024-09-15T21:17:41.377716+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:31.769758+0000","last_became_peered":"2024-09-15T21:17:31.769758+0000","last_unstale":"2024-09-15T21:17:41.377716+0000","last_undegraded":"2024-09-15T21:17:31.761602+0000","last_fullsized":"2024-09-15T21:17:31.761167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":56,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.376566+0000","last_change":"2024-09-15T21:17:44.421370+0000","last_active":"2024-09-15T21:17:44.421370+0000","last_peered":"2024-09-15T21:17:41.376566+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:31.763729+0000","last_became_peered":"2024-09-15T21:17:31.763729+0000","last_unstale":"2024-09-15T21:17:41.376566+0000","last_undegraded":"2024-09-15T21:17:31.759690+0000","last_fullsized":"2024-09-15T21:17:31.759441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":49,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.396261+0000","last_change":"2024-09-15T21:17:44.422041+0000","last_active":"2024-09-15T21:17:44.422041+0000","last_peered":"2024-09-15T21:17:41.396261+0000","last_clean":"2024-09-15T21:17:10.184528+0000","last_became_active":"2024-09-15T21:17:41.394639+0000","last_became_peered":"2024-09-15T21:17:41.394639+0000","last_unstale":"2024-09-15T21:17:41.396261+0000","last_undegraded":"2024-09-15T21:17:41.377894+0000","last_fullsized":"2024-09-15T21:17:41.396261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":57,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.400306+0000","last_change":"2024-09-15T21:17:44.422061+0000","last_active":"2024-09-15T21:17:44.422061+0000","last_peered":"2024-09-15T21:17:41.400306+0000","last_clean":"2024-09-15T21:17:41.400306+0000","last_became_active":"2024-09-15T21:17:41.395579+0000","last_became_peered":"2024-09-15T21:17:41.395579+0000","last_unstale":"2024-09-15T21:17:41.400306+0000","last_undegraded":"2024-09-15T21:17:41.400306+0000","last_fullsized":"2024-09-15T21:17:41.400306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":51,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.394662+0000","last_change":"2024-09-15T21:17:44.420518+0000","last_active":"2024-09-15T21:17:44.420518+0000","last_peered":"2024-09-15T21:17:41.394662+0000","last_clean":"2024-09-15T21:17:10.790830+0000","last_became_active":"2024-09-15T21:17:41.392812+0000","last_became_peered":"2024-09-15T21:17:41.392812+0000","last_unstale":"2024-09-15T21:17:41.394662+0000","last_undegraded":"2024-09-15T21:17:41.382237+0000","last_fullsized":"2024-09-15T21:17:41.394662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":49,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.378928+0000","last_change":"2024-09-15T21:17:44.422761+0000","last_active":"2024-09-15T21:17:44.422761+0000","last_peered":"2024-09-15T21:17:41.378928+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:31.766483+0000","last_became_peered":"2024-09-15T21:17:31.766483+0000","last_unstale":"2024-09-15T21:17:41.378928+0000","last_undegraded":"2024-09-15T21:17:31.763486+0000","last_fullsized":"2024-09-15T21:17:31.763327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.376640+0000","last_change":"2024-09-15T21:17:44.423796+0000","last_active":"2024-09-15T21:17:44.423796+0000","last_peered":"2024-09-15T21:17:41.376640+0000","last_clean":"2024-09-15T21:16:59.439373+0000","last_became_active":"2024-09-15T21:17:31.769273+0000","last_became_peered":"2024-09-15T21:17:31.769273+0000","last_unstale":"2024-09-15T21:17:41.376640+0000","last_undegraded":"2024-09-15T21:17:41.376640+0000","last_fullsized":"2024-09-15T21:17:31.760853+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":54,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.378004+0000","last_change":"2024-09-15T21:17:44.422183+0000","last_active":"2024-09-15T21:17:44.422183+0000","last_peered":"2024-09-15T21:17:41.378004+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:31.769321+0000","last_became_peered":"2024-09-15T21:17:31.769321+0000","last_unstale":"2024-09-15T21:17:41.378004+0000","last_undegraded":"2024-09-15T21:17:31.764181+0000","last_fullsized":"2024-09-15T21:17:31.763676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.400295+0000","last_change":"2024-09-15T21:17:44.422098+0000","last_active":"2024-09-15T21:17:44.422098+0000","last_peered":"2024-09-15T21:17:41.400295+0000","last_clean":"2024-09-15T21:17:41.400295+0000","last_became_active":"2024-09-15T21:17:41.395349+0000","last_became_peered":"2024-09-15T21:17:41.395349+0000","last_unstale":"2024-09-15T21:17:41.400295+0000","last_undegraded":"2024-09-15T21:17:41.400295+0000","last_fullsized":"2024-09-15T21:17:41.400295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":50,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.380104+0000","last_change":"2024-09-15T21:17:44.424812+0000","last_active":"2024-09-15T21:17:44.424812+0000","last_peered":"2024-09-15T21:17:41.380104+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:31.770073+0000","last_became_peered":"2024-09-15T21:17:31.770073+0000","last_unstale":"2024-09-15T21:17:41.380104+0000","last_undegraded":"2024-09-15T21:17:31.765001+0000","last_fullsized":"2024-09-15T21:17:31.764744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.377337+0000","last_change":"2024-09-15T21:17:44.422624+0000","last_active":"2024-09-15T21:17:44.422624+0000","last_peered":"2024-09-15T21:17:41.377337+0000","last_clean":"2024-09-15T21:16:59.440795+0000","last_became_active":"2024-09-15T21:17:31.766073+0000","last_became_peered":"2024-09-15T21:17:31.766073+0000","last_unstale":"2024-09-15T21:17:41.377337+0000","last_undegraded":"2024-09-15T21:17:41.377337+0000","last_fullsized":"2024-09-15T21:17:31.762569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":34,"state":"remapped+peering","last_fresh":"2024-09-15T21:17:41.399916+0000","last_change":"2024-09-15T21:17:44.420528+0000","last_active":"2024-09-15T21:17:44.420528+0000","last_peered":"2024-09-15T21:17:41.399916+0000","last_clean":"2024-09-15T21:17:41.399916+0000","last_became_active":"2024-09-15T21:17:41.394614+0000","last_became_peered":"2024-09-15T21:17:41.394614+0000","last_unstale":"2024-09-15T21:17:41.399916+0000","last_undegraded":"2024-09-15T21:17:41.399916+0000","last_fullsized":"2024-09-15T21:17:41.399916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":58,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-15T21:17:41.376841+0000","last_change":"2024-09-15T21:17:44.422509+0000","last_active":"2024-09-15T21:17:44.422509+0000","last_peered":"2024-09-15T21:17:41.376841+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:31.765165+0000","last_became_peered":"2024-09-15T21:17:31.765165+0000","last_unstale":"2024-09-15T21:17:41.376841+0000","last_undegraded":"2024-09-15T21:17:31.760261+0000","last_fullsized":"2024-09-15T21:17:31.759904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953475,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":288,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":294912,"data_stored":91178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28176045},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149703,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":339968,"data_stored":97367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":28110513},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:47.085 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:17:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:47.417 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435059,"num_in_osds":2,"osd_in_since":1726435064,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56971264,"bytes_avail":193216557056,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:17:38.286268+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:17:47.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '84ba6ec4-065b-47f6-87f1-d15da93be3d3', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 80, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435059, 'num_in_osds': 2, 'osd_in_since': 1726435064, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56971264, 'bytes_avail': 193216557056, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:16:26:965950+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:17:38.286268+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:17:50.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:17:50.439 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:17:50.433+0000 7fefb4d4f640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-15T21:17:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:50.751 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:17:50.770 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-15T21:17:50.290444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55668,"kb_used_data":644,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688012,"statfs":{"total":193273528320,"available":193216524288,"internally_reserved":0,"allocated":659456,"data_stored":197092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":56286558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001744"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434362+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:17:46.434362+0000","last_peered":"2024-09-15T21:17:46.434362+0000","last_clean":"2024-09-15T21:17:46.434362+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:17:46.434362+0000","last_undegraded":"2024-09-15T21:17:46.434362+0000","last_fullsized":"2024-09-15T21:17:46.434362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"24'4","reported_seq":49,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:17:46.452164+0000","last_change":"2024-09-15T21:17:46.452540+0000","last_active":"2024-09-15T21:17:46.452164+0000","last_peered":"2024-09-15T21:17:46.452164+0000","last_clean":"2024-09-15T21:17:10.403563+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:17:46.452164+0000","last_undegraded":"2024-09-15T21:17:46.441358+0000","last_fullsized":"2024-09-15T21:17:46.452164+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"24'3","reported_seq":65,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:46.453243+0000","last_change":"2024-09-15T21:17:48.378620+0000","last_active":"2024-09-15T21:17:46.453243+0000","last_peered":"2024-09-15T21:17:46.453243+0000","last_clean":"2024-09-15T21:17:11.463614+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:17:46.453243+0000","last_undegraded":"2024-09-15T21:17:46.453243+0000","last_fullsized":"2024-09-15T21:17:46.453243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.447488+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:17:46.447488+0000","last_peered":"2024-09-15T21:17:46.447488+0000","last_clean":"2024-09-15T21:17:46.447488+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:17:46.447488+0000","last_undegraded":"2024-09-15T21:17:46.447488+0000","last_fullsized":"2024-09-15T21:17:46.447488+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"24'5","reported_seq":66,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:17:46.434593+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:17:46.434593+0000","last_peered":"2024-09-15T21:17:46.434593+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:17:46.434593+0000","last_undegraded":"2024-09-15T21:17:46.434593+0000","last_fullsized":"2024-09-15T21:17:46.434593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"24'3","reported_seq":60,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433373+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:17:46.433373+0000","last_peered":"2024-09-15T21:17:46.433373+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:17:46.433373+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:17:46.433373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"24'4","reported_seq":62,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433428+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:17:46.433428+0000","last_peered":"2024-09-15T21:17:46.433428+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:17:46.433428+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:17:46.433428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"24'1","reported_seq":62,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433084+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:17:46.433084+0000","last_peered":"2024-09-15T21:17:46.433084+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:17:46.433084+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:17:46.433084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"24'5","reported_seq":67,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:46.448643+0000","last_change":"2024-09-15T21:17:48.378520+0000","last_active":"2024-09-15T21:17:46.448643+0000","last_peered":"2024-09-15T21:17:46.448643+0000","last_clean":"2024-09-15T21:17:11.507438+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:17:46.448643+0000","last_undegraded":"2024-09-15T21:17:46.448643+0000","last_fullsized":"2024-09-15T21:17:46.448643+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"24'6","reported_seq":57,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433659+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:17:46.433659+0000","last_peered":"2024-09-15T21:17:46.433659+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:17:46.433659+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:17:46.433659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"24'3","reported_seq":53,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:17:46.451349+0000","last_change":"2024-09-15T21:17:46.451722+0000","last_active":"2024-09-15T21:17:46.451349+0000","last_peered":"2024-09-15T21:17:46.451349+0000","last_clean":"2024-09-15T21:17:10.184528+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:17:46.451349+0000","last_undegraded":"2024-09-15T21:17:46.439313+0000","last_fullsized":"2024-09-15T21:17:46.451349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"24'4","reported_seq":55,"reported_epoch":37,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:17:46.450317+0000","last_change":"2024-09-15T21:17:46.450711+0000","last_active":"2024-09-15T21:17:46.450317+0000","last_peered":"2024-09-15T21:17:46.450317+0000","last_clean":"2024-09-15T21:17:10.790830+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:17:46.450317+0000","last_undegraded":"2024-09-15T21:17:46.439397+0000","last_fullsized":"2024-09-15T21:17:46.450317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"24'1","reported_seq":57,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433921+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:17:46.433921+0000","last_peered":"2024-09-15T21:17:46.433921+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:17:46.433921+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:17:46.433921+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"24'5","reported_seq":62,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433748+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:17:46.433748+0000","last_peered":"2024-09-15T21:17:46.433748+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:17:46.433748+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:17:46.433748+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"24'2","reported_seq":58,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.434097+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:17:46.434097+0000","last_peered":"2024-09-15T21:17:46.434097+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:17:46.434097+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:17:46.434097+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"24'9","reported_seq":66,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433895+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:17:46.433895+0000","last_peered":"2024-09-15T21:17:46.433895+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:17:46.433895+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:17:46.433895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953476,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":304,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":311296,"data_stored":96806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28176045},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149704,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":348160,"data_stored":100286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":28110513},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:17:50.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:17:51.126 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:51.126 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 52/60 objects degraded (86.667%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435059,"num_in_osds":2,"osd_in_since":1726435064,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":57004032,"bytes_avail":193216524288,"bytes_total":193273528320,"degraded_objects":52,"degraded_total":60,"degraded_ratio":0.8666666666666667,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:17:38.286268+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:17:51.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '84ba6ec4-065b-47f6-87f1-d15da93be3d3', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 52/60 objects degraded (86.667%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 84, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435059, 'num_in_osds': 2, 'osd_in_since': 1726435064, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 57004032, 'bytes_avail': 193216524288, 'bytes_total': 193273528320, 'degraded_objects': 52, 'degraded_total': 60, 'degraded_ratio': 0.8666666666666667, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:16:26:965950+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:17:38.286268+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:17:51.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-15T21:17:51.145 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:17:51.145 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:17:51.306 INFO:teuthology.orchestra.run.smithi114.stdout:111669149705 2024-09-15T21:17:51.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:51.307 INFO:teuthology.orchestra.run.smithi114.stdout:137438953477 2024-09-15T21:17:51.308 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:17:51.644 INFO:teuthology.orchestra.run.smithi114.stdout:137438953476 2024-09-15T21:17:51.645 INFO:teuthology.orchestra.run.smithi114.stdout:111669149704 2024-09-15T21:17:51.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953477 got 137438953476 for osd.2 2024-09-15T21:17:51.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149705 got 111669149704 for osd.0 2024-09-15T21:17:52.667 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:17:52.668 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:17:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:137438953477 2024-09-15T21:17:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:111669149706 2024-09-15T21:17:52.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953477 got 137438953477 for osd.2 2024-09-15T21:17:52.986 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:52.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149705 got 111669149706 for osd.0 2024-09-15T21:17:52.996 DEBUG:teuthology.parallel:result is None 2024-09-15T21:17:52.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:17:53.403 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:17:53.404 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 52/60 objects degraded (86.667%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":37,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435059,"num_in_osds":2,"osd_in_since":1726435064,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":57016320,"bytes_avail":193216512000,"bytes_total":193273528320,"degraded_objects":52,"degraded_total":60,"degraded_ratio":0.8666666666666667,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:17:38.286268+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:17:53.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '84ba6ec4-065b-47f6-87f1-d15da93be3d3', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 52/60 objects degraded (86.667%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 86, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 37, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435059, 'num_in_osds': 2, 'osd_in_since': 1726435064, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 57016320, 'bytes_avail': 193216512000, 'bytes_total': 193273528320, 'degraded_objects': 52, 'degraded_total': 60, 'degraded_ratio': 0.8666666666666667, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:16:26:965950+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:17:38.286268+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:17:53.422 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-15T21:17:53.422 DEBUG:teuthology.orchestra.run.smithi114:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-15T21:17:53.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:hints = 1 2024-09-15T21:17:53.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-15T21:17:53.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Object prefix: benchmark_data_smithi114_33359 2024-09-15T21:17:53.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:17:53.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 0 0 0 0 0 0 - 0 2024-09-15T21:17:54.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 1 20 1231 1211 4.72995 4.73047 0.00270576 0.00751568 2024-09-15T21:17:55.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 2 20 2980 2960 5.78066 6.83203 0.00595256 0.0134763 2024-09-15T21:17:56.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 3 20 5793 5773 7.51621 10.9883 0.00858073 0.0103772 2024-09-15T21:17:57.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 4 20 7805 7785 7.60185 7.85938 0.00742974 0.0102653 2024-09-15T21:17:58.468 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 5 20 9719 9699 7.57664 7.47656 0.00614256 0.0102999 2024-09-15T21:17:59.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 6 20 11513 11493 7.48175 7.00781 0.00656769 0.0104308 2024-09-15T21:18:00.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 7 20 13700 13680 7.63327 8.54297 0.0147084 0.0102226 2024-09-15T21:18:01.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 8 20 15358 15338 7.48861 6.47656 0.0147301 0.0104197 2024-09-15T21:18:02.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 9 20 16626 16606 7.20682 4.95312 0.00809371 0.0106806 2024-09-15T21:18:03.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:03.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 10 20 18406 18386 7.18139 6.95312 0.0131668 0.0108692 2024-09-15T21:18:03.925 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:03.928 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:03.956 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-15T21:18:02.294446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54371215,"num_objects":13300,"num_object_clones":0,"num_object_copies":26600,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":13300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13327,"num_write_kb":53128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13328,"ondisk_log_size":13328,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":176220,"kb_used_data":90684,"kb_used_omap":8,"kb_used_meta":85495,"kb_avail":188567460,"statfs":{"total":193273528320,"available":193093079040,"internally_reserved":0,"allocated":92860416,"data_stored":92392569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":87547230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,7,7,10,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54353920,"num_objects":13270,"num_object_clones":0,"num_object_copies":26540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13270,"num_write_kb":53080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004001"},"pg_stats":[{"pgid":"2.f","version":"37'970","reported_seq":1021,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:01.563923+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:01.563923+0000","last_peered":"2024-09-15T21:18:01.563923+0000","last_clean":"2024-09-15T21:18:01.563923+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:01.563923+0000","last_undegraded":"2024-09-15T21:18:01.563923+0000","last_fullsized":"2024-09-15T21:18:01.563923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3973120,"num_objects":970,"num_object_clones":0,"num_object_copies":1940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":970,"num_write_kb":3880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'611","reported_seq":656,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:58.338306+0000","last_change":"2024-09-15T21:17:53.476349+0000","last_active":"2024-09-15T21:17:58.338306+0000","last_peered":"2024-09-15T21:17:58.338306+0000","last_clean":"2024-09-15T21:17:10.403563+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:17:58.338306+0000","last_undegraded":"2024-09-15T21:17:46.452540+0000","last_fullsized":"2024-09-15T21:17:58.338306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2487506,"num_objects":609,"num_object_clones":0,"num_object_copies":1218,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":609}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'596","reported_seq":658,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:58.371020+0000","last_change":"2024-09-15T21:17:48.378620+0000","last_active":"2024-09-15T21:17:58.371020+0000","last_peered":"2024-09-15T21:17:58.371020+0000","last_clean":"2024-09-15T21:17:11.463614+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:17:58.371020+0000","last_undegraded":"2024-09-15T21:17:46.453243+0000","last_fullsized":"2024-09-15T21:17:58.371020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429545,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":596,"num_write_kb":2374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":594}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'607","reported_seq":657,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:58.380431+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:17:58.380431+0000","last_peered":"2024-09-15T21:17:58.380431+0000","last_clean":"2024-09-15T21:17:58.380431+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:17:58.380431+0000","last_undegraded":"2024-09-15T21:17:58.380431+0000","last_fullsized":"2024-09-15T21:17:58.380431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2486272,"num_objects":607,"num_object_clones":0,"num_object_copies":1214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":607,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'969","reported_seq":1028,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:01.541172+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:01.541172+0000","last_peered":"2024-09-15T21:18:01.541172+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:01.541172+0000","last_undegraded":"2024-09-15T21:18:01.541172+0000","last_fullsized":"2024-09-15T21:18:01.541172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3949778,"num_objects":966,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":3860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'988","reported_seq":1045,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.561378+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:01.561378+0000","last_peered":"2024-09-15T21:18:01.561378+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:01.561378+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:01.561378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4035794,"num_objects":987,"num_object_clones":0,"num_object_copies":1974,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":988,"num_write_kb":3943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'963","reported_seq":1021,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.557861+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:01.557861+0000","last_peered":"2024-09-15T21:18:01.557861+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:01.557861+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:01.557861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3929298,"num_objects":961,"num_object_clones":0,"num_object_copies":1922,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'982","reported_seq":1043,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.563714+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:01.563714+0000","last_peered":"2024-09-15T21:18:01.563714+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:01.563714+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:01.563714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4018793,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":982,"num_write_kb":3925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":982}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'556","reported_seq":618,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:58.376592+0000","last_change":"2024-09-15T21:17:48.378520+0000","last_active":"2024-09-15T21:17:58.376592+0000","last_peered":"2024-09-15T21:17:58.376592+0000","last_clean":"2024-09-15T21:17:11.507438+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:17:58.376592+0000","last_undegraded":"2024-09-15T21:17:46.448643+0000","last_fullsized":"2024-09-15T21:17:58.376592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2258130,"num_objects":553,"num_object_clones":0,"num_object_copies":1106,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":556,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":553}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'966","reported_seq":1016,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.570569+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:01.570569+0000","last_peered":"2024-09-15T21:18:01.570569+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:01.570569+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:01.570569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3933394,"num_objects":962,"num_object_clones":0,"num_object_copies":1924,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":966,"num_write_kb":3844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":962}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'581","reported_seq":631,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:58.370995+0000","last_change":"2024-09-15T21:17:53.485687+0000","last_active":"2024-09-15T21:17:58.370995+0000","last_peered":"2024-09-15T21:17:58.370995+0000","last_clean":"2024-09-15T21:17:10.184528+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:17:58.370995+0000","last_undegraded":"2024-09-15T21:17:46.451722+0000","last_fullsized":"2024-09-15T21:17:58.370995+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2368722,"num_objects":580,"num_object_clones":0,"num_object_copies":1160,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":581,"num_write_kb":2315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'598","reported_seq":649,"reported_epoch":37,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:17:58.376583+0000","last_change":"2024-09-15T21:17:53.474003+0000","last_active":"2024-09-15T21:17:58.376583+0000","last_peered":"2024-09-15T21:17:58.376583+0000","last_clean":"2024-09-15T21:17:10.790830+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:17:58.376583+0000","last_undegraded":"2024-09-15T21:17:46.450711+0000","last_fullsized":"2024-09-15T21:17:58.376583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2434258,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":598,"num_write_kb":2379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'979","reported_seq":1034,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.570842+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:01.570842+0000","last_peered":"2024-09-15T21:18:01.570842+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:01.570842+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:01.570842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":979,"log_dups_size":0,"ondisk_log_size":979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4006505,"num_objects":979,"num_object_clones":0,"num_object_copies":1958,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":979,"num_write_kb":3913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":979}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'989","reported_seq":1045,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.557167+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:01.557167+0000","last_peered":"2024-09-15T21:18:01.557167+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:01.557167+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:01.557167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4032315,"num_objects":987,"num_object_clones":0,"num_object_copies":1974,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":989,"num_write_kb":3940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'926","reported_seq":982,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.551526+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:01.551526+0000","last_peered":"2024-09-15T21:18:01.551526+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:01.551526+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:01.551526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3785938,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":926,"num_write_kb":3698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":926}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1044","reported_seq":1101,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:01.571803+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:01.571803+0000","last_peered":"2024-09-15T21:18:01.571803+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:01.571803+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:01.571803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1044,"log_dups_size":0,"ondisk_log_size":1044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4241828,"num_objects":1039,"num_object_clones":0,"num_object_copies":2078,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":54371196,"num_objects":13298,"num_object_clones":0,"num_object_copies":26596,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":13298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13325,"num_write_kb":53126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102805504,"data_stored":102673302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13325,"ondisk_log_size":13325,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953479,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72704,"kb_used_data":32688,"kb_used_omap":4,"kb_used_meta":39995,"kb_avail":94299136,"statfs":{"total":96636764160,"available":96562315264,"internally_reserved":0,"allocated":33472512,"data_stored":33258022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":40955565},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,2,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149708,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103516,"kb_used_data":57996,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94268324,"statfs":{"total":96636764160,"available":96530763776,"internally_reserved":0,"allocated":59387904,"data_stored":59134547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":46591665},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,5,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63729664,"data_stored":63694874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38961152,"data_stored":38961152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:03.957 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-15T21:18:03.957 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-15T21:18:03.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:04.176 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_4", 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:04.177 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1224592654, 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'1", 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:04.178 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_6", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3101443614, 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'2", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:04.179 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "NotRecovering", 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:04.180 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:04.211 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-15T21:18:04.212 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-15T21:18:04.393 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:04.408 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-15T21:18:04.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-15T21:18:04.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 11 20 19882 19862 7.05264 5.76562 0.00278729 0.0109438 2024-09-15T21:18:04.528 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:04.528 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 1, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 1, 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:04.529 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_7", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 2358579165, 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:04.530 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "NotRecovering", 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:04.531 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:04.544 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-15T21:18:04.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-15T21:18:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:18:04.894 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-15T21:18:04.894 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovering 2024-09-15T21:18:04.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-15T21:18:05.016 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:05.016 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.017 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_3", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 943960171, 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'5", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_1", 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 2885884443, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'1", 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.019 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:05.020 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:05.030 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-15T21:18:05.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-15T21:18:05.158 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:05.173 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-15T21:18:05.173 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:05.296 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:05.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_9", 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1217469130, 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'1", 2024-09-15T21:18:05.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_9", 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 170713514, 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'2", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.300 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.301 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:05.301 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:05.310 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-15T21:18:05.310 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-15T21:18:05.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 12 20 19994 19974 6.50137 0.4375 0.0076836 0.0117518 2024-09-15T21:18:05.656 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:05.702 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-15T21:18:05.702 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:05.849 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_5", 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 2998549577, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'2", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_2", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 4032093161, 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:05.851 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'1", 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:05.852 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:05.853 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:05.864 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-15T21:18:05.864 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-15T21:18:06.066 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:06.093 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-15T21:18:06.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-15T21:18:06.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 13 20 20966 20946 6.29331 3.79688 0.00779291 0.0124074 2024-09-15T21:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:06.516 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 1, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 1, 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_8", 2024-09-15T21:18:06.517 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 2730327960, 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'1", 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:06.518 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:06.519 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:06.532 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-15T21:18:06.532 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-15T21:18:06.657 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:18:06.672 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-15T21:18:06.672 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-15T21:18:06.672 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:06.858 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_8", 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1901787523, 2024-09-15T21:18:06.859 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'5", 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'3", 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:06.860 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_1", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3122379363, 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:06.861 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "NotRecovering", 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:06.862 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:06.886 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-15T21:18:06.887 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-15T21:18:07.025 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:07.062 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-15T21:18:07.062 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-15T21:18:07.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:07.263 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_6", 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1708673474, 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:07.264 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'5", 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_9", 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.266 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 831402034, 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'6", 2024-09-15T21:18:07.267 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:07.268 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:07.269 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:07.270 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:07.270 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:07.287 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-15T21:18:07.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-15T21:18:07.469 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 14 20 21991 21971 6.12976 4.00391 0.0114807 0.0127356 2024-09-15T21:18:07.513 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:07.546 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-15T21:18:07.546 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-15T21:18:07.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_3", 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 459768081, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'1", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_3", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 480221105, 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'2", 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "NotRecovering", 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:07.773 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:07.796 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-15T21:18:07.796 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-15T21:18:08.053 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:08.082 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-15T21:18:08.082 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-15T21:18:08.082 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_5", 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 4248945696, 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_5", 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 4212789968, 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "NotRecovering", 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:08.330 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:08.354 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-15T21:18:08.355 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-15T21:18:08.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 15 20 23376 23356 6.08176 5.41016 0.00176932 0.0127541 2024-09-15T21:18:08.497 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:08.512 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-15T21:18:08.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-15T21:18:08.641 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 1, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 1, 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_7", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3761017527, 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'1", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:08.644 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:08.655 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-15T21:18:08.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-15T21:18:08.812 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:18:08.849 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-15T21:18:08.849 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-15T21:18:08.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-15T21:18:09.075 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:09.075 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:09.075 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:09.075 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.075 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 3, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 3, 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:09.076 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_8", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1861424966, 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'5", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:09.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_6", 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3910131350, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'4", 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:09.078 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_4", 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 2787839990, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.079 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'3", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:09.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:09.081 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:09.081 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:09.081 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:09.107 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-15T21:18:09.108 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-15T21:18:09.345 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-15T21:18:09.385 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-15T21:18:09.385 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-15T21:18:09.385 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-15T21:18:09.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 16 20 24453 24433 5.96457 4.20703 0.00841038 0.0130905 2024-09-15T21:18:09.594 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 2, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 2, 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:09.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_1", 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 26281365, 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'1", 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:09.596 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "new_2", 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3196340981, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:09.597 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'2", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:09.598 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:09.600 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:09.600 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:09.625 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-15T21:18:09.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-15T21:18:09.861 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:18:09.888 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-15T21:18:09.888 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-15T21:18:09.889 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-15T21:18:09.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "offset": { 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "", 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": 0, 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 0, 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": -9223372036854775808, 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:10.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "num_missing": 4, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "num_unfound": 3, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "objects": [ 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_4", 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 873891716, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:10.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'8", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existing_7", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 3996155460, 2024-09-15T21:18:10.097 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'9", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "22'6", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": { 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "oid": "existed_2", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "key": "", 2024-09-15T21:18:10.098 INFO:teuthology.orchestra.run.smithi114.stdout: "snapid": -2, 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "hash": 1767667508, 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 0, 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "pool": 2, 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "namespace": "" 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "need": "24'7", 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "have": "0'0", 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": "none", 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: "locations": [] 2024-09-15T21:18:10.099 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "Active", 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "available_might_have_unfound": true, 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "might_have_unfound": [ 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "osd": "1", 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "osd is down" 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:10.100 INFO:teuthology.orchestra.run.smithi114.stdout: "more": false 2024-09-15T21:18:10.101 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:10.121 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-15T21:18:10.121 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-15T21:18:10.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 17 20 25912 25892 5.94893 5.69922 0.00593115 0.0131246 2024-09-15T21:18:10.578 INFO:teuthology.orchestra.run.smithi114.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-15T21:18:10.603 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-15T21:18:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-15T21:18:10.857 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-15T21:18:11.051 INFO:teuthology.orchestra.run.smithi114.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-15T21:18:11.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:18:11.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:18:11.208 INFO:teuthology.orchestra.run.smithi114.stdout:137438953482 2024-09-15T21:18:11.208 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:18:11.209 INFO:teuthology.orchestra.run.smithi114.stdout:111669149710 2024-09-15T21:18:11.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:18:11.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 18 20 26841 26821 5.82001 3.62891 0.00908359 0.0132665 2024-09-15T21:18:11.597 INFO:teuthology.orchestra.run.smithi114.stdout:111669149709 2024-09-15T21:18:11.597 INFO:teuthology.orchestra.run.smithi114.stdout:137438953480 2024-09-15T21:18:11.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149710 got 111669149709 for osd.0 2024-09-15T21:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953482 got 137438953480 for osd.2 2024-09-15T21:18:12.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 19 20 28453 28433 5.84508 6.29688 0.00913926 0.0133585 2024-09-15T21:18:12.629 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:18:12.635 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:18:13.296 INFO:teuthology.orchestra.run.smithi114.stdout:111669149711 2024-09-15T21:18:13.304 INFO:teuthology.orchestra.run.smithi114.stdout:137438953482 2024-09-15T21:18:13.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 111669149710 got 111669149711 for osd.0 2024-09-15T21:18:13.339 DEBUG:teuthology.parallel:result is None 2024-09-15T21:18:13.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 137438953482 got 137438953482 for osd.2 2024-09-15T21:18:13.347 DEBUG:teuthology.parallel:result is None 2024-09-15T21:18:13.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-15T21:18:13.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:13.472 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:18:13.469549+0000 min lat: 0.00128075 max lat: 0.831995 avg lat: 0.0133897 2024-09-15T21:18:13.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:18:13.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 20 20 29686 29666 5.79362 4.81641 0.0169649 0.0133897 2024-09-15T21:18:13.667 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:13.668 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:13.683 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-15T21:18:12.297036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110046686,"num_objects":26885,"num_object_clones":0,"num_object_copies":53770,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":79540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26949,"ondisk_log_size":26949,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":338492,"kb_used_data":211292,"kb_used_omap":8,"kb_used_meta":127159,"kb_avail":188405188,"statfs":{"total":193273528320,"available":192926912512,"internally_reserved":0,"allocated":216363008,"data_stored":215909077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":130211166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,8,8,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80419407,"num_objects":19626,"num_object_clones":0,"num_object_copies":39252,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":19626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":50576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"2.f","version":"37'1674","reported_seq":1724,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:11.565263+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:11.565263+0000","last_peered":"2024-09-15T21:18:11.565263+0000","last_clean":"2024-09-15T21:18:11.565263+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:11.565263+0000","last_undegraded":"2024-09-15T21:18:11.565263+0000","last_fullsized":"2024-09-15T21:18:11.565263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1674,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1678","reported_seq":1755,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.985528+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:10.985528+0000","last_peered":"2024-09-15T21:18:10.985528+0000","last_clean":"2024-09-15T21:18:10.985528+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:10.985528+0000","last_undegraded":"2024-09-15T21:18:10.985528+0000","last_fullsized":"2024-09-15T21:18:10.985528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6848512,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":404,"num_write_kb":1616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'1647","reported_seq":1739,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996435+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:10.996435+0000","last_peered":"2024-09-15T21:18:10.996435+0000","last_clean":"2024-09-15T21:18:10.996435+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:10.996435+0000","last_undegraded":"2024-09-15T21:18:10.996435+0000","last_fullsized":"2024-09-15T21:18:10.996435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":1647,"log_dups_size":0,"ondisk_log_size":1647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":370,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1704","reported_seq":1754,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996126+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:10.996126+0000","last_peered":"2024-09-15T21:18:10.996126+0000","last_clean":"2024-09-15T21:18:10.996126+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:10.996126+0000","last_undegraded":"2024-09-15T21:18:10.996126+0000","last_fullsized":"2024-09-15T21:18:10.996126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1704,"log_dups_size":0,"ondisk_log_size":1704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6979584,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1720","reported_seq":1779,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:11.574209+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:11.574209+0000","last_peered":"2024-09-15T21:18:11.574209+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:11.574209+0000","last_undegraded":"2024-09-15T21:18:11.574209+0000","last_fullsized":"2024-09-15T21:18:11.574209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1720,"log_dups_size":0,"ondisk_log_size":1720,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7017682,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1715}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1708","reported_seq":1764,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.539781+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:11.539781+0000","last_peered":"2024-09-15T21:18:11.539781+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:11.539781+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:11.539781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6976722,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":6815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1690","reported_seq":1747,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.564168+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:11.564168+0000","last_peered":"2024-09-15T21:18:11.564168+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:11.564168+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:11.564168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1690,"log_dups_size":0,"ondisk_log_size":1690,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1698","reported_seq":1759,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.568366+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:11.568366+0000","last_peered":"2024-09-15T21:18:11.568366+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:11.568366+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:11.568366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1698,"log_dups_size":0,"ondisk_log_size":1698,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6947433,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1697,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1650","reported_seq":1747,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.991927+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:10.991927+0000","last_peered":"2024-09-15T21:18:10.991927+0000","last_clean":"2024-09-15T21:18:10.991927+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:10.991927+0000","last_undegraded":"2024-09-15T21:18:10.991927+0000","last_fullsized":"2024-09-15T21:18:10.991927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":1650,"log_dups_size":0,"ondisk_log_size":1650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1669","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.561739+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:11.561739+0000","last_peered":"2024-09-15T21:18:11.561739+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:11.561739+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:11.561739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6804690,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'1639","reported_seq":1723,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.972414+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:10.972414+0000","last_peered":"2024-09-15T21:18:10.972414+0000","last_clean":"2024-09-15T21:18:10.972414+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:10.972414+0000","last_undegraded":"2024-09-15T21:18:10.972414+0000","last_fullsized":"2024-09-15T21:18:10.972414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":1639,"log_dups_size":0,"ondisk_log_size":1639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1680","reported_seq":1767,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996344+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:10.996344+0000","last_peered":"2024-09-15T21:18:10.996344+0000","last_clean":"2024-09-15T21:18:10.996344+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:10.996344+0000","last_undegraded":"2024-09-15T21:18:10.996344+0000","last_fullsized":"2024-09-15T21:18:10.996344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":1680,"log_dups_size":0,"ondisk_log_size":1680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1672","reported_seq":1727,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.560503+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:11.560503+0000","last_peered":"2024-09-15T21:18:11.560503+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:11.560503+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:11.560503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1672,"log_dups_size":0,"ondisk_log_size":1672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6840937,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1671,"num_write_kb":6681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1685","reported_seq":1740,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.536229+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:11.536229+0000","last_peered":"2024-09-15T21:18:11.536229+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:11.536229+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:11.536229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6870843,"num_objects":1680,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1682,"num_write_kb":6712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1664","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.551849+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:11.551849+0000","last_peered":"2024-09-15T21:18:11.551849+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:11.551849+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:11.551849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6800594,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1662,"num_write_kb":6642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1768","reported_seq":1823,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.567043+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:11.567043+0000","last_peered":"2024-09-15T21:18:11.567043+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:11.567043+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:11.567043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1768,"log_dups_size":0,"ondisk_log_size":1768,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7195044,"num_objects":1760,"num_object_clones":0,"num_object_copies":3520,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":110046667,"num_objects":26883,"num_object_clones":0,"num_object_copies":53766,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19913,"num_write_kb":79538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220106752,"data_stored":219988466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26946,"ondisk_log_size":26946,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169032,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94202808,"statfs":{"total":96636764160,"available":96463675392,"internally_reserved":0,"allocated":107929600,"data_stored":107715110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":65138349},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169460,"kb_used_data":105892,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94202380,"statfs":{"total":96636764160,"available":96463237120,"internally_reserved":0,"allocated":108433408,"data_stored":108193967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":65072817},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110505984,"data_stored":110485110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109486080,"data_stored":109486080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:13.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:13.965 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:13.965 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:13.981 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-15T21:18:12.297036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110046686,"num_objects":26885,"num_object_clones":0,"num_object_copies":53770,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":79540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26949,"ondisk_log_size":26949,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":338492,"kb_used_data":211292,"kb_used_omap":8,"kb_used_meta":127159,"kb_avail":188405188,"statfs":{"total":193273528320,"available":192926912512,"internally_reserved":0,"allocated":216363008,"data_stored":215909077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":130211166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,8,8,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80419407,"num_objects":19626,"num_object_clones":0,"num_object_copies":39252,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":19626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":50576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"2.f","version":"37'1674","reported_seq":1724,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:11.565263+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:11.565263+0000","last_peered":"2024-09-15T21:18:11.565263+0000","last_clean":"2024-09-15T21:18:11.565263+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:11.565263+0000","last_undegraded":"2024-09-15T21:18:11.565263+0000","last_fullsized":"2024-09-15T21:18:11.565263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1674,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1678","reported_seq":1755,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.985528+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:10.985528+0000","last_peered":"2024-09-15T21:18:10.985528+0000","last_clean":"2024-09-15T21:18:10.985528+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:10.985528+0000","last_undegraded":"2024-09-15T21:18:10.985528+0000","last_fullsized":"2024-09-15T21:18:10.985528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6848512,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":404,"num_write_kb":1616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'1647","reported_seq":1739,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996435+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:10.996435+0000","last_peered":"2024-09-15T21:18:10.996435+0000","last_clean":"2024-09-15T21:18:10.996435+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:10.996435+0000","last_undegraded":"2024-09-15T21:18:10.996435+0000","last_fullsized":"2024-09-15T21:18:10.996435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":1647,"log_dups_size":0,"ondisk_log_size":1647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":370,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1704","reported_seq":1754,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996126+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:10.996126+0000","last_peered":"2024-09-15T21:18:10.996126+0000","last_clean":"2024-09-15T21:18:10.996126+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:10.996126+0000","last_undegraded":"2024-09-15T21:18:10.996126+0000","last_fullsized":"2024-09-15T21:18:10.996126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1704,"log_dups_size":0,"ondisk_log_size":1704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6979584,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1720","reported_seq":1779,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:11.574209+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:11.574209+0000","last_peered":"2024-09-15T21:18:11.574209+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:11.574209+0000","last_undegraded":"2024-09-15T21:18:11.574209+0000","last_fullsized":"2024-09-15T21:18:11.574209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1720,"log_dups_size":0,"ondisk_log_size":1720,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7017682,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1715}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1708","reported_seq":1764,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.539781+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:11.539781+0000","last_peered":"2024-09-15T21:18:11.539781+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:11.539781+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:11.539781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6976722,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":6815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1690","reported_seq":1747,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.564168+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:11.564168+0000","last_peered":"2024-09-15T21:18:11.564168+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:11.564168+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:11.564168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1690,"log_dups_size":0,"ondisk_log_size":1690,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1698","reported_seq":1759,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.568366+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:11.568366+0000","last_peered":"2024-09-15T21:18:11.568366+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:11.568366+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:11.568366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1698,"log_dups_size":0,"ondisk_log_size":1698,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6947433,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1697,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1650","reported_seq":1747,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.991927+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:10.991927+0000","last_peered":"2024-09-15T21:18:10.991927+0000","last_clean":"2024-09-15T21:18:10.991927+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:10.991927+0000","last_undegraded":"2024-09-15T21:18:10.991927+0000","last_fullsized":"2024-09-15T21:18:10.991927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":1650,"log_dups_size":0,"ondisk_log_size":1650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1669","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.561739+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:11.561739+0000","last_peered":"2024-09-15T21:18:11.561739+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:11.561739+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:11.561739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6804690,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'1639","reported_seq":1723,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.972414+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:10.972414+0000","last_peered":"2024-09-15T21:18:10.972414+0000","last_clean":"2024-09-15T21:18:10.972414+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:10.972414+0000","last_undegraded":"2024-09-15T21:18:10.972414+0000","last_fullsized":"2024-09-15T21:18:10.972414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":1639,"log_dups_size":0,"ondisk_log_size":1639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1680","reported_seq":1767,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996344+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:10.996344+0000","last_peered":"2024-09-15T21:18:10.996344+0000","last_clean":"2024-09-15T21:18:10.996344+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:10.996344+0000","last_undegraded":"2024-09-15T21:18:10.996344+0000","last_fullsized":"2024-09-15T21:18:10.996344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":1680,"log_dups_size":0,"ondisk_log_size":1680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1672","reported_seq":1727,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.560503+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:11.560503+0000","last_peered":"2024-09-15T21:18:11.560503+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:11.560503+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:11.560503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1672,"log_dups_size":0,"ondisk_log_size":1672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6840937,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1671,"num_write_kb":6681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1685","reported_seq":1740,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.536229+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:11.536229+0000","last_peered":"2024-09-15T21:18:11.536229+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:11.536229+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:11.536229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6870843,"num_objects":1680,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1682,"num_write_kb":6712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1664","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.551849+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:11.551849+0000","last_peered":"2024-09-15T21:18:11.551849+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:11.551849+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:11.551849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6800594,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1662,"num_write_kb":6642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1768","reported_seq":1823,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.567043+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:11.567043+0000","last_peered":"2024-09-15T21:18:11.567043+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:11.567043+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:11.567043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1768,"log_dups_size":0,"ondisk_log_size":1768,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7195044,"num_objects":1760,"num_object_clones":0,"num_object_copies":3520,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":110046667,"num_objects":26883,"num_object_clones":0,"num_object_copies":53766,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19913,"num_write_kb":79538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220106752,"data_stored":219988466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26946,"ondisk_log_size":26946,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169032,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94202808,"statfs":{"total":96636764160,"available":96463675392,"internally_reserved":0,"allocated":107929600,"data_stored":107715110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":65138349},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169460,"kb_used_data":105892,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94202380,"statfs":{"total":96636764160,"available":96463237120,"internally_reserved":0,"allocated":108433408,"data_stored":108193967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":65072817},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110505984,"data_stored":110485110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109486080,"data_stored":109486080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:13.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:14.256 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:14.256 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:14.272 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-15T21:18:12.297036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110046686,"num_objects":26885,"num_object_clones":0,"num_object_copies":53770,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":79540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26949,"ondisk_log_size":26949,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":338492,"kb_used_data":211292,"kb_used_omap":8,"kb_used_meta":127159,"kb_avail":188405188,"statfs":{"total":193273528320,"available":192926912512,"internally_reserved":0,"allocated":216363008,"data_stored":215909077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":130211166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,8,8,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80419407,"num_objects":19626,"num_object_clones":0,"num_object_copies":39252,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":19626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12629,"num_write_kb":50576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"2.f","version":"37'1674","reported_seq":1724,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:11.565263+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:11.565263+0000","last_peered":"2024-09-15T21:18:11.565263+0000","last_clean":"2024-09-15T21:18:11.565263+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:11.565263+0000","last_undegraded":"2024-09-15T21:18:11.565263+0000","last_fullsized":"2024-09-15T21:18:11.565263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1674,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1678","reported_seq":1755,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.985528+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:10.985528+0000","last_peered":"2024-09-15T21:18:10.985528+0000","last_clean":"2024-09-15T21:18:10.985528+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:10.985528+0000","last_undegraded":"2024-09-15T21:18:10.985528+0000","last_fullsized":"2024-09-15T21:18:10.985528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6848512,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":404,"num_write_kb":1616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'1647","reported_seq":1739,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996435+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:10.996435+0000","last_peered":"2024-09-15T21:18:10.996435+0000","last_clean":"2024-09-15T21:18:10.996435+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:10.996435+0000","last_undegraded":"2024-09-15T21:18:10.996435+0000","last_fullsized":"2024-09-15T21:18:10.996435+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":1647,"log_dups_size":0,"ondisk_log_size":1647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":370,"num_write_kb":1480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1704","reported_seq":1754,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996126+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:10.996126+0000","last_peered":"2024-09-15T21:18:10.996126+0000","last_clean":"2024-09-15T21:18:10.996126+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:10.996126+0000","last_undegraded":"2024-09-15T21:18:10.996126+0000","last_fullsized":"2024-09-15T21:18:10.996126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1704,"log_dups_size":0,"ondisk_log_size":1704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6979584,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1704,"num_write_kb":6816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1720","reported_seq":1779,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:11.574209+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:11.574209+0000","last_peered":"2024-09-15T21:18:11.574209+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:11.574209+0000","last_undegraded":"2024-09-15T21:18:11.574209+0000","last_fullsized":"2024-09-15T21:18:11.574209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1720,"log_dups_size":0,"ondisk_log_size":1720,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7017682,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1715}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1708","reported_seq":1764,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.539781+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:11.539781+0000","last_peered":"2024-09-15T21:18:11.539781+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:11.539781+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:11.539781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6976722,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":6815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1690","reported_seq":1747,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.564168+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:11.564168+0000","last_peered":"2024-09-15T21:18:11.564168+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:11.564168+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:11.564168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1690,"log_dups_size":0,"ondisk_log_size":1690,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1698","reported_seq":1759,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.568366+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:11.568366+0000","last_peered":"2024-09-15T21:18:11.568366+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:11.568366+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:11.568366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1698,"log_dups_size":0,"ondisk_log_size":1698,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6947433,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1697,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1650","reported_seq":1747,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.991927+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:10.991927+0000","last_peered":"2024-09-15T21:18:10.991927+0000","last_clean":"2024-09-15T21:18:10.991927+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:10.991927+0000","last_undegraded":"2024-09-15T21:18:10.991927+0000","last_fullsized":"2024-09-15T21:18:10.991927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":1650,"log_dups_size":0,"ondisk_log_size":1650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1669","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.561739+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:11.561739+0000","last_peered":"2024-09-15T21:18:11.561739+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:11.561739+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:11.561739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6804690,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'1639","reported_seq":1723,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.972414+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:10.972414+0000","last_peered":"2024-09-15T21:18:10.972414+0000","last_clean":"2024-09-15T21:18:10.972414+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:10.972414+0000","last_undegraded":"2024-09-15T21:18:10.972414+0000","last_fullsized":"2024-09-15T21:18:10.972414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":1639,"log_dups_size":0,"ondisk_log_size":1639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1680","reported_seq":1767,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:10.996344+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:10.996344+0000","last_peered":"2024-09-15T21:18:10.996344+0000","last_clean":"2024-09-15T21:18:10.996344+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:10.996344+0000","last_undegraded":"2024-09-15T21:18:10.996344+0000","last_fullsized":"2024-09-15T21:18:10.996344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":1680,"log_dups_size":0,"ondisk_log_size":1680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1672","reported_seq":1727,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.560503+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:11.560503+0000","last_peered":"2024-09-15T21:18:11.560503+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:11.560503+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:11.560503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1672,"log_dups_size":0,"ondisk_log_size":1672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6840937,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1671,"num_write_kb":6681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1685","reported_seq":1740,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.536229+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:11.536229+0000","last_peered":"2024-09-15T21:18:11.536229+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:11.536229+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:11.536229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6870843,"num_objects":1680,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1682,"num_write_kb":6712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1664","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.551849+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:11.551849+0000","last_peered":"2024-09-15T21:18:11.551849+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:11.551849+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:11.551849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6800594,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1662,"num_write_kb":6642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1768","reported_seq":1823,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.567043+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:11.567043+0000","last_peered":"2024-09-15T21:18:11.567043+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:11.567043+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:11.567043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1768,"log_dups_size":0,"ondisk_log_size":1768,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7195044,"num_objects":1760,"num_object_clones":0,"num_object_copies":3520,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":110046667,"num_objects":26883,"num_object_clones":0,"num_object_copies":53766,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19913,"num_write_kb":79538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220106752,"data_stored":219988466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26946,"ondisk_log_size":26946,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953482,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169032,"kb_used_data":105400,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94202808,"statfs":{"total":96636764160,"available":96463675392,"internally_reserved":0,"allocated":107929600,"data_stored":107715110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":65138349},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169460,"kb_used_data":105892,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94202380,"statfs":{"total":96636764160,"available":96463237120,"internally_reserved":0,"allocated":108433408,"data_stored":108193967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":65072817},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110505984,"data_stored":110485110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109486080,"data_stored":109486080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:14.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 21 20 29686 29666 5.51772 0 - 0.0133897 2024-09-15T21:18:15.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 22 20 30433 30413 5.39954 1.45898 0.021652 0.0144601 2024-09-15T21:18:16.470 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 23 20 31709 31689 5.38147 4.98438 0.00859072 0.0145105 2024-09-15T21:18:17.274 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:17.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 24 20 32575 32555 5.29819 3.38281 0.0151182 0.0147389 2024-09-15T21:18:17.685 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:17.685 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:17.712 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-15T21:18:16.298041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114531806,"num_objects":27980,"num_object_clones":0,"num_object_copies":55960,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21010,"num_write_kb":83920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28044,"ondisk_log_size":28044,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":352828,"kb_used_data":221468,"kb_used_omap":8,"kb_used_meta":131319,"kb_avail":188390852,"statfs":{"total":193273528320,"available":192912232448,"internally_reserved":0,"allocated":226783232,"data_stored":226329301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":134471006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,8,8,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46664271,"num_objects":11385,"num_object_clones":0,"num_object_copies":22770,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":11385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"2.f","version":"37'1674","reported_seq":1724,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:11.565263+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:11.565263+0000","last_peered":"2024-09-15T21:18:11.565263+0000","last_clean":"2024-09-15T21:18:11.565263+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:11.565263+0000","last_undegraded":"2024-09-15T21:18:11.565263+0000","last_fullsized":"2024-09-15T21:18:11.565263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1674,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1868","reported_seq":1944,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.306113+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:13.306113+0000","last_peered":"2024-09-15T21:18:13.306113+0000","last_clean":"2024-09-15T21:18:13.306113+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:13.306113+0000","last_undegraded":"2024-09-15T21:18:13.306113+0000","last_fullsized":"2024-09-15T21:18:13.306113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":1868,"log_dups_size":0,"ondisk_log_size":1868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":594,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'1830","reported_seq":1923,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.320041+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:13.320041+0000","last_peered":"2024-09-15T21:18:13.320041+0000","last_clean":"2024-09-15T21:18:13.320041+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:13.320041+0000","last_undegraded":"2024-09-15T21:18:13.320041+0000","last_fullsized":"2024-09-15T21:18:13.320041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":1830,"log_dups_size":0,"ondisk_log_size":1830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7479296,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1908","reported_seq":1958,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.305131+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:13.305131+0000","last_peered":"2024-09-15T21:18:13.305131+0000","last_clean":"2024-09-15T21:18:13.305131+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:13.305131+0000","last_undegraded":"2024-09-15T21:18:13.305131+0000","last_fullsized":"2024-09-15T21:18:13.305131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1908,"log_dups_size":0,"ondisk_log_size":1908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7815168,"num_objects":1908,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1908,"num_write_kb":7632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1720","reported_seq":1779,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:11.574209+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:11.574209+0000","last_peered":"2024-09-15T21:18:11.574209+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:11.574209+0000","last_undegraded":"2024-09-15T21:18:11.574209+0000","last_fullsized":"2024-09-15T21:18:11.574209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1720,"log_dups_size":0,"ondisk_log_size":1720,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7017682,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1715}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1708","reported_seq":1764,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.539781+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:11.539781+0000","last_peered":"2024-09-15T21:18:11.539781+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:11.539781+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:11.539781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6976722,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":6815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1690","reported_seq":1747,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.564168+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:11.564168+0000","last_peered":"2024-09-15T21:18:11.564168+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:11.564168+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:11.564168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1690,"log_dups_size":0,"ondisk_log_size":1690,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1698","reported_seq":1759,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.568366+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:11.568366+0000","last_peered":"2024-09-15T21:18:11.568366+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:11.568366+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:11.568366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1698,"log_dups_size":0,"ondisk_log_size":1698,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6947433,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1697,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1808","reported_seq":1905,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.283479+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:13.283479+0000","last_peered":"2024-09-15T21:18:13.283479+0000","last_clean":"2024-09-15T21:18:13.283479+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:13.283479+0000","last_undegraded":"2024-09-15T21:18:13.283479+0000","last_fullsized":"2024-09-15T21:18:13.283479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":1808,"log_dups_size":0,"ondisk_log_size":1808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7376896,"num_objects":1801,"num_object_clones":0,"num_object_copies":3602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":394,"num_write_kb":1576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1669","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.561739+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:11.561739+0000","last_peered":"2024-09-15T21:18:11.561739+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:11.561739+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:11.561739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6804690,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'1807","reported_seq":1891,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.314481+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:13.314481+0000","last_peered":"2024-09-15T21:18:13.314481+0000","last_clean":"2024-09-15T21:18:13.314481+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:13.314481+0000","last_undegraded":"2024-09-15T21:18:13.314481+0000","last_fullsized":"2024-09-15T21:18:13.314481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":1807,"log_dups_size":0,"ondisk_log_size":1807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7380992,"num_objects":1802,"num_object_clones":0,"num_object_copies":3604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":358,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1872","reported_seq":1958,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.336962+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:13.336962+0000","last_peered":"2024-09-15T21:18:13.336962+0000","last_clean":"2024-09-15T21:18:13.336962+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:13.336962+0000","last_undegraded":"2024-09-15T21:18:13.336962+0000","last_fullsized":"2024-09-15T21:18:13.336962+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":1872,"log_dups_size":0,"ondisk_log_size":1872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7643136,"num_objects":1866,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":1084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1672","reported_seq":1727,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.560503+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:11.560503+0000","last_peered":"2024-09-15T21:18:11.560503+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:11.560503+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:11.560503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1672,"log_dups_size":0,"ondisk_log_size":1672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6840937,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1671,"num_write_kb":6681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1685","reported_seq":1740,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.536229+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:11.536229+0000","last_peered":"2024-09-15T21:18:11.536229+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:11.536229+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:11.536229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6870843,"num_objects":1680,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1682,"num_write_kb":6712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1664","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.551849+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:11.551849+0000","last_peered":"2024-09-15T21:18:11.551849+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:11.551849+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:11.551849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6800594,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1662,"num_write_kb":6642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1768","reported_seq":1823,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.567043+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:11.567043+0000","last_peered":"2024-09-15T21:18:11.567043+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:11.567043+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:11.567043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1768,"log_dups_size":0,"ondisk_log_size":1768,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7195044,"num_objects":1760,"num_object_clones":0,"num_object_copies":3520,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":114531787,"num_objects":27978,"num_object_clones":0,"num_object_copies":55956,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21008,"num_write_kb":83918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":232161280,"data_stored":232042994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28041,"ondisk_log_size":28041,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953483,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183368,"kb_used_data":115576,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94188472,"statfs":{"total":96636764160,"available":96448995328,"internally_reserved":0,"allocated":118349824,"data_stored":118135334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":69398189},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169460,"kb_used_data":105892,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94202380,"statfs":{"total":96636764160,"available":96463237120,"internally_reserved":0,"allocated":108433408,"data_stored":108193967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":65072817},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110505984,"data_stored":110485110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121540608,"data_stored":121540608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:17.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:18.287 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:18.287 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-15T21:18:16.298041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114531806,"num_objects":27980,"num_object_clones":0,"num_object_copies":55960,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21010,"num_write_kb":83920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28044,"ondisk_log_size":28044,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":352828,"kb_used_data":221468,"kb_used_omap":8,"kb_used_meta":131319,"kb_avail":188390852,"statfs":{"total":193273528320,"available":192912232448,"internally_reserved":0,"allocated":226783232,"data_stored":226329301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":134471006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,8,8,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46664271,"num_objects":11385,"num_object_clones":0,"num_object_copies":22770,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":11385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4388,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"2.f","version":"37'1674","reported_seq":1724,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:11.565263+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:11.565263+0000","last_peered":"2024-09-15T21:18:11.565263+0000","last_clean":"2024-09-15T21:18:11.565263+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:11.565263+0000","last_undegraded":"2024-09-15T21:18:11.565263+0000","last_fullsized":"2024-09-15T21:18:11.565263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1674,"log_dups_size":0,"ondisk_log_size":1674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6856704,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1674,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'1868","reported_seq":1944,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.306113+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:13.306113+0000","last_peered":"2024-09-15T21:18:13.306113+0000","last_clean":"2024-09-15T21:18:13.306113+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:13.306113+0000","last_undegraded":"2024-09-15T21:18:13.306113+0000","last_fullsized":"2024-09-15T21:18:13.306113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":1868,"log_dups_size":0,"ondisk_log_size":1868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":594,"num_write_kb":2376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'1830","reported_seq":1923,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.320041+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:13.320041+0000","last_peered":"2024-09-15T21:18:13.320041+0000","last_clean":"2024-09-15T21:18:13.320041+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:13.320041+0000","last_undegraded":"2024-09-15T21:18:13.320041+0000","last_fullsized":"2024-09-15T21:18:13.320041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":1830,"log_dups_size":0,"ondisk_log_size":1830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7479296,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'1908","reported_seq":1958,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.305131+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:13.305131+0000","last_peered":"2024-09-15T21:18:13.305131+0000","last_clean":"2024-09-15T21:18:13.305131+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:13.305131+0000","last_undegraded":"2024-09-15T21:18:13.305131+0000","last_fullsized":"2024-09-15T21:18:13.305131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1908,"log_dups_size":0,"ondisk_log_size":1908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7815168,"num_objects":1908,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1908,"num_write_kb":7632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'1720","reported_seq":1779,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:11.574209+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:11.574209+0000","last_peered":"2024-09-15T21:18:11.574209+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:11.574209+0000","last_undegraded":"2024-09-15T21:18:11.574209+0000","last_fullsized":"2024-09-15T21:18:11.574209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1720,"log_dups_size":0,"ondisk_log_size":1720,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7017682,"num_objects":1715,"num_object_clones":0,"num_object_copies":3430,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1718,"num_write_kb":6856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1715}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'1708","reported_seq":1764,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.539781+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:11.539781+0000","last_peered":"2024-09-15T21:18:11.539781+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:11.539781+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:11.539781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1708,"log_dups_size":0,"ondisk_log_size":1708,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6976722,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1706,"num_write_kb":6815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1705}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'1690","reported_seq":1747,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.564168+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:11.564168+0000","last_peered":"2024-09-15T21:18:11.564168+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:11.564168+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:11.564168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1690,"log_dups_size":0,"ondisk_log_size":1690,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'1698","reported_seq":1759,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.568366+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:11.568366+0000","last_peered":"2024-09-15T21:18:11.568366+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:11.568366+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:11.568366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1698,"log_dups_size":0,"ondisk_log_size":1698,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6947433,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1697,"num_write_kb":6785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'1808","reported_seq":1905,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.283479+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:13.283479+0000","last_peered":"2024-09-15T21:18:13.283479+0000","last_clean":"2024-09-15T21:18:13.283479+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:13.283479+0000","last_undegraded":"2024-09-15T21:18:13.283479+0000","last_fullsized":"2024-09-15T21:18:13.283479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":1808,"log_dups_size":0,"ondisk_log_size":1808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7376896,"num_objects":1801,"num_object_clones":0,"num_object_copies":3602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":394,"num_write_kb":1576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1669","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.561739+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:11.561739+0000","last_peered":"2024-09-15T21:18:11.561739+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:11.561739+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:11.561739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1669,"log_dups_size":0,"ondisk_log_size":1669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6804690,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1667,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'1807","reported_seq":1891,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.314481+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:13.314481+0000","last_peered":"2024-09-15T21:18:13.314481+0000","last_clean":"2024-09-15T21:18:13.314481+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:13.314481+0000","last_undegraded":"2024-09-15T21:18:13.314481+0000","last_fullsized":"2024-09-15T21:18:13.314481+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":1807,"log_dups_size":0,"ondisk_log_size":1807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7380992,"num_objects":1802,"num_object_clones":0,"num_object_copies":3604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":358,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'1872","reported_seq":1958,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:13.336962+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:13.336962+0000","last_peered":"2024-09-15T21:18:13.336962+0000","last_clean":"2024-09-15T21:18:13.336962+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:13.336962+0000","last_undegraded":"2024-09-15T21:18:13.336962+0000","last_fullsized":"2024-09-15T21:18:13.336962+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":1872,"log_dups_size":0,"ondisk_log_size":1872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7643136,"num_objects":1866,"num_object_clones":0,"num_object_copies":3732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":1084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1672","reported_seq":1727,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.560503+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:11.560503+0000","last_peered":"2024-09-15T21:18:11.560503+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:11.560503+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:11.560503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1672,"log_dups_size":0,"ondisk_log_size":1672,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6840937,"num_objects":1671,"num_object_clones":0,"num_object_copies":3342,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1671,"num_write_kb":6681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1685","reported_seq":1740,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.536229+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:11.536229+0000","last_peered":"2024-09-15T21:18:11.536229+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:11.536229+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:11.536229+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1685,"log_dups_size":0,"ondisk_log_size":1685,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6870843,"num_objects":1680,"num_object_clones":0,"num_object_copies":3360,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1682,"num_write_kb":6712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1664","reported_seq":1719,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.551849+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:11.551849+0000","last_peered":"2024-09-15T21:18:11.551849+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:11.551849+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:11.551849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6800594,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1662,"num_write_kb":6642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'1768","reported_seq":1823,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:11.567043+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:11.567043+0000","last_peered":"2024-09-15T21:18:11.567043+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:11.567043+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:11.567043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1768,"log_dups_size":0,"ondisk_log_size":1768,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7195044,"num_objects":1760,"num_object_clones":0,"num_object_copies":3520,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1765,"num_write_kb":7031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1759}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":114531787,"num_objects":27978,"num_object_clones":0,"num_object_copies":55956,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21008,"num_write_kb":83918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":232161280,"data_stored":232042994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28041,"ondisk_log_size":28041,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953483,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183368,"kb_used_data":115576,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94188472,"statfs":{"total":96636764160,"available":96448995328,"internally_reserved":0,"allocated":118349824,"data_stored":118135334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":69398189},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,7,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149711,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169460,"kb_used_data":105892,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94202380,"statfs":{"total":96636764160,"available":96463237120,"internally_reserved":0,"allocated":108433408,"data_stored":108193967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":65072817},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,11,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":110505984,"data_stored":110485110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":121540608,"data_stored":121540608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:18.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 25 20 33890 33870 5.29171 5.13672 0.00962929 0.0147557 2024-09-15T21:18:19.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 26 20 35397 35377 5.31458 5.88672 0.0265367 0.0146931 2024-09-15T21:18:20.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 27 20 36261 36241 5.24272 3.375 0.0141427 0.0148963 2024-09-15T21:18:21.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:21.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 28 20 37655 37635 5.24994 5.44531 0.0133869 0.0148736 2024-09-15T21:18:21.895 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:21.896 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:21.913 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-15T21:18:20.299113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133246430,"num_objects":32549,"num_object_clones":0,"num_object_copies":65098,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25579,"num_write_kb":102196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32613,"ondisk_log_size":32613,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":431784,"kb_used_data":256008,"kb_used_omap":8,"kb_used_meta":175735,"kb_avail":188311896,"statfs":{"total":193273528320,"available":192831381504,"internally_reserved":0,"allocated":262152192,"data_stored":261690705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":179952990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51333711,"num_objects":12525,"num_object_clones":0,"num_object_copies":25050,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-21,"num_objects_dirty":12525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5528,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003075"},"pg_stats":[{"pgid":"2.f","version":"37'1972","reported_seq":2023,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:16.478001+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:16.478001+0000","last_peered":"2024-09-15T21:18:16.478001+0000","last_clean":"2024-09-15T21:18:16.478001+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:16.478001+0000","last_undegraded":"2024-09-15T21:18:16.478001+0000","last_fullsized":"2024-09-15T21:18:16.478001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1972,"log_dups_size":0,"ondisk_log_size":1972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8077312,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1972,"num_write_kb":7888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2141","reported_seq":2217,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.383783+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:18.383783+0000","last_peered":"2024-09-15T21:18:18.383783+0000","last_clean":"2024-09-15T21:18:18.383783+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:18.383783+0000","last_undegraded":"2024-09-15T21:18:18.383783+0000","last_fullsized":"2024-09-15T21:18:18.383783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":2141,"log_dups_size":0,"ondisk_log_size":2141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":867,"num_write_kb":3468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2086","reported_seq":2179,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.380201+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:18.380201+0000","last_peered":"2024-09-15T21:18:18.380201+0000","last_clean":"2024-09-15T21:18:18.380201+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:18.380201+0000","last_undegraded":"2024-09-15T21:18:18.380201+0000","last_fullsized":"2024-09-15T21:18:18.380201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":2086,"log_dups_size":0,"ondisk_log_size":2086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2173","reported_seq":2223,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.360446+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:18.360446+0000","last_peered":"2024-09-15T21:18:18.360446+0000","last_clean":"2024-09-15T21:18:18.360446+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:18.360446+0000","last_undegraded":"2024-09-15T21:18:18.360446+0000","last_fullsized":"2024-09-15T21:18:18.360446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2173,"log_dups_size":0,"ondisk_log_size":2173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8900608,"num_objects":2173,"num_object_clones":0,"num_object_copies":4346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2173,"num_write_kb":8692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2034","reported_seq":2094,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:16.480662+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:16.480662+0000","last_peered":"2024-09-15T21:18:16.480662+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:16.480662+0000","last_undegraded":"2024-09-15T21:18:16.480662+0000","last_fullsized":"2024-09-15T21:18:16.480662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2034,"log_dups_size":0,"ondisk_log_size":2034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8303826,"num_objects":2029,"num_object_clones":0,"num_object_copies":4058,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2032,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2004","reported_seq":2060,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.486255+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:16.486255+0000","last_peered":"2024-09-15T21:18:16.486255+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:16.486255+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:16.486255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2004,"log_dups_size":0,"ondisk_log_size":2004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8189138,"num_objects":2001,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2002,"num_write_kb":7999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2001}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2004","reported_seq":2061,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.476765+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:16.476765+0000","last_peered":"2024-09-15T21:18:16.476765+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:16.476765+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:16.476765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2004,"log_dups_size":0,"ondisk_log_size":2004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8185042,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2002,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2008","reported_seq":2069,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.490821+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:16.490821+0000","last_peered":"2024-09-15T21:18:16.490821+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:16.490821+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:16.490821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2008,"log_dups_size":0,"ondisk_log_size":2008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8217193,"num_objects":2007,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2007,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2071","reported_seq":2166,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.380389+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:18.380389+0000","last_peered":"2024-09-15T21:18:18.380389+0000","last_clean":"2024-09-15T21:18:18.380389+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:18.380389+0000","last_undegraded":"2024-09-15T21:18:18.380389+0000","last_fullsized":"2024-09-15T21:18:18.380389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":2071,"log_dups_size":0,"ondisk_log_size":2071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":2064,"num_object_clones":0,"num_object_copies":4128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1944","reported_seq":1994,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.477508+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:16.477508+0000","last_peered":"2024-09-15T21:18:16.477508+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:16.477508+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:16.477508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1944,"log_dups_size":0,"ondisk_log_size":1944,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7931090,"num_objects":1938,"num_object_clones":0,"num_object_copies":3876,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1942,"num_write_kb":7748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2063","reported_seq":2146,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.385603+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:18.385603+0000","last_peered":"2024-09-15T21:18:18.385603+0000","last_clean":"2024-09-15T21:18:18.385603+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:18.385603+0000","last_undegraded":"2024-09-15T21:18:18.385603+0000","last_fullsized":"2024-09-15T21:18:18.385603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":2063,"log_dups_size":0,"ondisk_log_size":2063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8429568,"num_objects":2058,"num_object_clones":0,"num_object_copies":4116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":614,"num_write_kb":2456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2119","reported_seq":2206,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.388079+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:18.388079+0000","last_peered":"2024-09-15T21:18:18.388079+0000","last_clean":"2024-09-15T21:18:18.388079+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:18.388079+0000","last_undegraded":"2024-09-15T21:18:18.388079+0000","last_fullsized":"2024-09-15T21:18:18.388079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":2072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1987","reported_seq":2043,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.486311+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:16.486311+0000","last_peered":"2024-09-15T21:18:16.486311+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:16.486311+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:16.486311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1987,"log_dups_size":0,"ondisk_log_size":1987,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8131177,"num_objects":1986,"num_object_clones":0,"num_object_copies":3972,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1986,"num_write_kb":7941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1986}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1977","reported_seq":2032,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.488251+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:16.488251+0000","last_peered":"2024-09-15T21:18:16.488251+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:16.488251+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:16.488251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1977,"log_dups_size":0,"ondisk_log_size":1977,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8066875,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1974,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1972}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1963","reported_seq":2018,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.457750+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:16.457750+0000","last_peered":"2024-09-15T21:18:16.457750+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:16.457750+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:16.457750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1963,"log_dups_size":0,"ondisk_log_size":1963,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8025298,"num_objects":1961,"num_object_clones":0,"num_object_copies":3922,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":7838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2064","reported_seq":2119,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.481707+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:16.481707+0000","last_peered":"2024-09-15T21:18:16.481707+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:16.481707+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:16.481707+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2064,"log_dups_size":0,"ondisk_log_size":2064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8407460,"num_objects":2056,"num_object_clones":0,"num_object_copies":4112,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2061,"num_write_kb":8215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2055}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":133246411,"num_objects":32547,"num_object_clones":0,"num_object_copies":65094,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25577,"num_write_kb":102194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268472320,"data_stored":268350555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32610,"ondisk_log_size":32610,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953484,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221968,"kb_used_data":131968,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94149872,"statfs":{"total":96636764160,"available":96409468928,"internally_reserved":0,"allocated":135135232,"data_stored":134913186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":92139181},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209816,"kb_used_data":124040,"kb_used_omap":4,"kb_used_meta":85755,"kb_avail":94162024,"statfs":{"total":96636764160,"available":96421912576,"internally_reserved":0,"allocated":127016960,"data_stored":126777519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":87813809},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129994752,"data_stored":129973878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138362880,"data_stored":138359401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:21.915 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:22.265 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-15T21:18:20.299113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":133246430,"num_objects":32549,"num_object_clones":0,"num_object_copies":65098,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25579,"num_write_kb":102196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32613,"ondisk_log_size":32613,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":431784,"kb_used_data":256008,"kb_used_omap":8,"kb_used_meta":175735,"kb_avail":188311896,"statfs":{"total":193273528320,"available":192831381504,"internally_reserved":0,"allocated":262152192,"data_stored":261690705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":179952990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51333711,"num_objects":12525,"num_object_clones":0,"num_object_copies":25050,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-21,"num_objects_dirty":12525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5528,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003075"},"pg_stats":[{"pgid":"2.f","version":"37'1972","reported_seq":2023,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:16.478001+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:16.478001+0000","last_peered":"2024-09-15T21:18:16.478001+0000","last_clean":"2024-09-15T21:18:16.478001+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:16.478001+0000","last_undegraded":"2024-09-15T21:18:16.478001+0000","last_fullsized":"2024-09-15T21:18:16.478001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1972,"log_dups_size":0,"ondisk_log_size":1972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8077312,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1972,"num_write_kb":7888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2141","reported_seq":2217,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.383783+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:18.383783+0000","last_peered":"2024-09-15T21:18:18.383783+0000","last_clean":"2024-09-15T21:18:18.383783+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:18.383783+0000","last_undegraded":"2024-09-15T21:18:18.383783+0000","last_fullsized":"2024-09-15T21:18:18.383783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":2141,"log_dups_size":0,"ondisk_log_size":2141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":867,"num_write_kb":3468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2086","reported_seq":2179,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.380201+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:18.380201+0000","last_peered":"2024-09-15T21:18:18.380201+0000","last_clean":"2024-09-15T21:18:18.380201+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:18.380201+0000","last_undegraded":"2024-09-15T21:18:18.380201+0000","last_fullsized":"2024-09-15T21:18:18.380201+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":2086,"log_dups_size":0,"ondisk_log_size":2086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2173","reported_seq":2223,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.360446+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:18.360446+0000","last_peered":"2024-09-15T21:18:18.360446+0000","last_clean":"2024-09-15T21:18:18.360446+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:18.360446+0000","last_undegraded":"2024-09-15T21:18:18.360446+0000","last_fullsized":"2024-09-15T21:18:18.360446+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2173,"log_dups_size":0,"ondisk_log_size":2173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8900608,"num_objects":2173,"num_object_clones":0,"num_object_copies":4346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2173,"num_write_kb":8692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2034","reported_seq":2094,"reported_epoch":37,"state":"active+recovering","last_fresh":"2024-09-15T21:18:16.480662+0000","last_change":"2024-09-15T21:17:46.434593+0000","last_active":"2024-09-15T21:18:16.480662+0000","last_peered":"2024-09-15T21:18:16.480662+0000","last_clean":"2024-09-15T21:17:10.273369+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:16.480662+0000","last_undegraded":"2024-09-15T21:18:16.480662+0000","last_fullsized":"2024-09-15T21:18:16.480662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2034,"log_dups_size":0,"ondisk_log_size":2034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8303826,"num_objects":2029,"num_object_clones":0,"num_object_copies":4058,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2032,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2004","reported_seq":2060,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.486255+0000","last_change":"2024-09-15T21:17:45.443564+0000","last_active":"2024-09-15T21:18:16.486255+0000","last_peered":"2024-09-15T21:18:16.486255+0000","last_clean":"2024-09-15T21:17:11.722524+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:16.486255+0000","last_undegraded":"2024-09-15T21:17:45.433972+0000","last_fullsized":"2024-09-15T21:18:16.486255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2004,"log_dups_size":0,"ondisk_log_size":2004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8189138,"num_objects":2001,"num_object_clones":0,"num_object_copies":4002,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2002,"num_write_kb":7999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2001}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2004","reported_seq":2061,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.476765+0000","last_change":"2024-09-15T21:17:45.437785+0000","last_active":"2024-09-15T21:18:16.476765+0000","last_peered":"2024-09-15T21:18:16.476765+0000","last_clean":"2024-09-15T21:17:10.616371+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:16.476765+0000","last_undegraded":"2024-09-15T21:17:45.432576+0000","last_fullsized":"2024-09-15T21:18:16.476765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2004,"log_dups_size":0,"ondisk_log_size":2004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8185042,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2002,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2008","reported_seq":2069,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.490821+0000","last_change":"2024-09-15T21:17:45.437237+0000","last_active":"2024-09-15T21:18:16.490821+0000","last_peered":"2024-09-15T21:18:16.490821+0000","last_clean":"2024-09-15T21:16:56.405960+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:16.490821+0000","last_undegraded":"2024-09-15T21:17:45.428779+0000","last_fullsized":"2024-09-15T21:18:16.490821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2008,"log_dups_size":0,"ondisk_log_size":2008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8217193,"num_objects":2007,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2007,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2071","reported_seq":2166,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.380389+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:18.380389+0000","last_peered":"2024-09-15T21:18:18.380389+0000","last_clean":"2024-09-15T21:18:18.380389+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:18.380389+0000","last_undegraded":"2024-09-15T21:18:18.380389+0000","last_fullsized":"2024-09-15T21:18:18.380389+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":2071,"log_dups_size":0,"ondisk_log_size":2071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":2064,"num_object_clones":0,"num_object_copies":4128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":657,"num_write_kb":2628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'1944","reported_seq":1994,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.477508+0000","last_change":"2024-09-15T21:17:45.443544+0000","last_active":"2024-09-15T21:18:16.477508+0000","last_peered":"2024-09-15T21:18:16.477508+0000","last_clean":"2024-09-15T21:17:11.806090+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:16.477508+0000","last_undegraded":"2024-09-15T21:17:45.433502+0000","last_fullsized":"2024-09-15T21:18:16.477508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1944,"log_dups_size":0,"ondisk_log_size":1944,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7931090,"num_objects":1938,"num_object_clones":0,"num_object_copies":3876,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1942,"num_write_kb":7748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":64,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:17:46.433321+0000","last_change":"2024-09-15T21:17:45.437836+0000","last_active":"2024-09-15T21:17:46.433321+0000","last_peered":"2024-09-15T21:17:46.433321+0000","last_clean":"2024-09-15T21:17:23.622377+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:17:46.433321+0000","last_undegraded":"2024-09-15T21:17:45.432618+0000","last_fullsized":"2024-09-15T21:17:46.433321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2063","reported_seq":2146,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.385603+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:18.385603+0000","last_peered":"2024-09-15T21:18:18.385603+0000","last_clean":"2024-09-15T21:18:18.385603+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:18.385603+0000","last_undegraded":"2024-09-15T21:18:18.385603+0000","last_fullsized":"2024-09-15T21:18:18.385603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":2063,"log_dups_size":0,"ondisk_log_size":2063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8429568,"num_objects":2058,"num_object_clones":0,"num_object_copies":4116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":614,"num_write_kb":2456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2119","reported_seq":2206,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:18.388079+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:18.388079+0000","last_peered":"2024-09-15T21:18:18.388079+0000","last_clean":"2024-09-15T21:18:18.388079+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:18.388079+0000","last_undegraded":"2024-09-15T21:18:18.388079+0000","last_fullsized":"2024-09-15T21:18:18.388079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":2072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'1987","reported_seq":2043,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.486311+0000","last_change":"2024-09-15T21:17:45.443594+0000","last_active":"2024-09-15T21:18:16.486311+0000","last_peered":"2024-09-15T21:18:16.486311+0000","last_clean":"2024-09-15T21:16:59.440890+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:16.486311+0000","last_undegraded":"2024-09-15T21:17:45.433860+0000","last_fullsized":"2024-09-15T21:18:16.486311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1987,"log_dups_size":0,"ondisk_log_size":1987,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8131177,"num_objects":1986,"num_object_clones":0,"num_object_copies":3972,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1986,"num_write_kb":7941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1986}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'1977","reported_seq":2032,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.488251+0000","last_change":"2024-09-15T21:17:45.443353+0000","last_active":"2024-09-15T21:18:16.488251+0000","last_peered":"2024-09-15T21:18:16.488251+0000","last_clean":"2024-09-15T21:17:11.636945+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:16.488251+0000","last_undegraded":"2024-09-15T21:17:45.434090+0000","last_fullsized":"2024-09-15T21:18:16.488251+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1977,"log_dups_size":0,"ondisk_log_size":1977,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8066875,"num_objects":1972,"num_object_clones":0,"num_object_copies":3944,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1974,"num_write_kb":7880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1972}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'1963","reported_seq":2018,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.457750+0000","last_change":"2024-09-15T21:17:45.443339+0000","last_active":"2024-09-15T21:18:16.457750+0000","last_peered":"2024-09-15T21:18:16.457750+0000","last_clean":"2024-09-15T21:16:59.440015+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:16.457750+0000","last_undegraded":"2024-09-15T21:17:45.433451+0000","last_fullsized":"2024-09-15T21:18:16.457750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":1963,"log_dups_size":0,"ondisk_log_size":1963,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8025298,"num_objects":1961,"num_object_clones":0,"num_object_copies":3922,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":7838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2064","reported_seq":2119,"reported_epoch":37,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:18:16.481707+0000","last_change":"2024-09-15T21:17:45.437505+0000","last_active":"2024-09-15T21:18:16.481707+0000","last_peered":"2024-09-15T21:18:16.481707+0000","last_clean":"2024-09-15T21:17:11.091708+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:16.481707+0000","last_undegraded":"2024-09-15T21:17:45.432796+0000","last_fullsized":"2024-09-15T21:18:16.481707+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2064,"log_dups_size":0,"ondisk_log_size":2064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8407460,"num_objects":2056,"num_object_clones":0,"num_object_copies":4112,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2061,"num_write_kb":8215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2055}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":133246411,"num_objects":32547,"num_object_clones":0,"num_object_copies":65094,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25577,"num_write_kb":102194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268472320,"data_stored":268350555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32610,"ondisk_log_size":32610,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953484,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":221968,"kb_used_data":131968,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94149872,"statfs":{"total":96636764160,"available":96409468928,"internally_reserved":0,"allocated":135135232,"data_stored":134913186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":92139181},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149712,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209816,"kb_used_data":124040,"kb_used_omap":4,"kb_used_meta":85755,"kb_avail":94162024,"statfs":{"total":96636764160,"available":96421912576,"internally_reserved":0,"allocated":127016960,"data_stored":126777519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":87813809},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129994752,"data_stored":129973878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138362880,"data_stored":138359401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:22.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 29 20 38447 38427 5.17558 3.09375 0.0119511 0.0150882 2024-09-15T21:18:23.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 30 20 39247 39227 5.10722 3.125 0.0169171 0.0152895 2024-09-15T21:18:24.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 31 20 40234 40214 5.06683 3.85547 0.00882488 0.0154103 2024-09-15T21:18:25.267 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:25.471 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 32 20 41553 41533 5.06949 5.15234 0.0121265 0.0154017 2024-09-15T21:18:25.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:25.761 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:25.788 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-09-15T21:18:24.301474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156790632,"num_objects":38292,"num_object_clones":0,"num_object_copies":76584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24637,"num_write_kb":98476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38362,"ondisk_log_size":38362,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":477584,"kb_used_data":301808,"kb_used_omap":8,"kb_used_meta":175735,"kb_avail":188266096,"statfs":{"total":193273528320,"available":192784482304,"internally_reserved":0,"allocated":309051392,"data_stored":308607300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8866,"internal_metadata":179952990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46743946,"num_objects":11407,"num_object_clones":0,"num_object_copies":22814,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4722,"num_write_kb":18936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004437"},"pg_stats":[{"pgid":"2.f","version":"37'2337","reported_seq":2388,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.545730+0000","last_change":"2024-09-15T21:17:45.440568+0000","last_active":"2024-09-15T21:18:21.545730+0000","last_peered":"2024-09-15T21:18:21.545730+0000","last_clean":"2024-09-15T21:18:21.545730+0000","last_became_active":"2024-09-15T21:17:45.440301+0000","last_became_peered":"2024-09-15T21:17:45.440301+0000","last_unstale":"2024-09-15T21:18:21.545730+0000","last_undegraded":"2024-09-15T21:18:21.545730+0000","last_fullsized":"2024-09-15T21:18:21.545730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:15:55.022547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9572352,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2337,"num_write_kb":9348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'2441","reported_seq":2518,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.359833+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:23.359833+0000","last_peered":"2024-09-15T21:18:23.359833+0000","last_clean":"2024-09-15T21:18:23.359833+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:23.359833+0000","last_undegraded":"2024-09-15T21:18:23.359833+0000","last_fullsized":"2024-09-15T21:18:23.359833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":2441,"log_dups_size":0,"ondisk_log_size":2441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9973760,"num_objects":2435,"num_object_clones":0,"num_object_copies":4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1167,"num_write_kb":4668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"37'2435","reported_seq":2528,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.359927+0000","last_change":"2024-09-15T21:18:06.394220+0000","last_active":"2024-09-15T21:18:23.359927+0000","last_peered":"2024-09-15T21:18:23.359927+0000","last_clean":"2024-09-15T21:18:23.359927+0000","last_became_active":"2024-09-15T21:17:46.447368+0000","last_became_peered":"2024-09-15T21:17:46.447368+0000","last_unstale":"2024-09-15T21:18:23.359927+0000","last_undegraded":"2024-09-15T21:18:23.359927+0000","last_fullsized":"2024-09-15T21:18:23.359927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":2435,"log_dups_size":0,"ondisk_log_size":2435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:36.602061+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1158,"num_write_kb":4632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'2500","reported_seq":2550,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.361267+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:23.361267+0000","last_peered":"2024-09-15T21:18:23.361267+0000","last_clean":"2024-09-15T21:18:23.361267+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:23.361267+0000","last_undegraded":"2024-09-15T21:18:23.361267+0000","last_fullsized":"2024-09-15T21:18:23.361267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2500,"log_dups_size":0,"ondisk_log_size":2500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10240000,"num_objects":2500,"num_object_clones":0,"num_object_copies":5000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2500,"num_write_kb":10000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'2409","reported_seq":2508,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.553634+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:21.553634+0000","last_peered":"2024-09-15T21:18:21.553634+0000","last_clean":"2024-09-15T21:18:21.553634+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:21.553634+0000","last_undegraded":"2024-09-15T21:18:21.553634+0000","last_fullsized":"2024-09-15T21:18:21.553634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":2409,"log_dups_size":0,"ondisk_log_size":2409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9838592,"num_objects":2402,"num_object_clones":0,"num_object_copies":4804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":1068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"37'2356","reported_seq":2423,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.553884+0000","last_change":"2024-09-15T21:18:17.435554+0000","last_active":"2024-09-15T21:18:21.553884+0000","last_peered":"2024-09-15T21:18:21.553884+0000","last_clean":"2024-09-15T21:18:21.553884+0000","last_became_active":"2024-09-15T21:17:45.443195+0000","last_became_peered":"2024-09-15T21:17:45.443195+0000","last_unstale":"2024-09-15T21:18:21.553884+0000","last_undegraded":"2024-09-15T21:18:21.553884+0000","last_fullsized":"2024-09-15T21:18:21.553884+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9630930,"num_objects":2353,"num_object_clones":0,"num_object_copies":4706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"37'2404","reported_seq":2499,"reported_epoch":37,"state":"active+clean+scrubbing","last_fresh":"2024-09-15T21:18:21.585772+0000","last_change":"2024-09-15T21:18:20.755644+0000","last_active":"2024-09-15T21:18:21.585772+0000","last_peered":"2024-09-15T21:18:21.585772+0000","last_clean":"2024-09-15T21:18:21.585772+0000","last_became_active":"2024-09-15T21:17:45.437639+0000","last_became_peered":"2024-09-15T21:17:45.437639+0000","last_unstale":"2024-09-15T21:18:21.585772+0000","last_undegraded":"2024-09-15T21:18:21.585772+0000","last_fullsized":"2024-09-15T21:18:21.585772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":2349,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9823442,"num_objects":2400,"num_object_clones":0,"num_object_copies":4800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2402,"num_write_kb":9596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"37'2404","reported_seq":2504,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.553199+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:21.553199+0000","last_peered":"2024-09-15T21:18:21.553199+0000","last_clean":"2024-09-15T21:18:21.553199+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:21.553199+0000","last_undegraded":"2024-09-15T21:18:21.553199+0000","last_fullsized":"2024-09-15T21:18:21.553199+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9838592,"num_objects":2402,"num_object_clones":0,"num_object_copies":4804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.432951+0000","last_change":"2024-09-15T21:17:45.437353+0000","last_active":"2024-09-15T21:17:46.432951+0000","last_peered":"2024-09-15T21:17:46.432951+0000","last_clean":"2024-09-15T21:17:46.432951+0000","last_became_active":"2024-09-15T21:17:45.437093+0000","last_became_peered":"2024-09-15T21:17:45.437093+0000","last_unstale":"2024-09-15T21:17:46.432951+0000","last_undegraded":"2024-09-15T21:17:46.432951+0000","last_fullsized":"2024-09-15T21:17:46.432951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:55.059074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"37'2426","reported_seq":2523,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.392443+0000","last_change":"2024-09-15T21:18:08.359844+0000","last_active":"2024-09-15T21:18:23.392443+0000","last_peered":"2024-09-15T21:18:23.392443+0000","last_clean":"2024-09-15T21:18:23.392443+0000","last_became_active":"2024-09-15T21:17:46.443741+0000","last_became_peered":"2024-09-15T21:17:46.443741+0000","last_unstale":"2024-09-15T21:18:23.392443+0000","last_undegraded":"2024-09-15T21:18:23.392443+0000","last_fullsized":"2024-09-15T21:18:23.392443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":2426,"log_dups_size":0,"ondisk_log_size":2426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:48.813236+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9908224,"num_objects":2419,"num_object_clones":0,"num_object_copies":4838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1012,"num_write_kb":4048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.446199+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:17:46.446199+0000","last_peered":"2024-09-15T21:17:46.446199+0000","last_clean":"2024-09-15T21:17:46.446199+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:17:46.446199+0000","last_undegraded":"2024-09-15T21:17:46.446199+0000","last_fullsized":"2024-09-15T21:17:46.446199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"37'2343","reported_seq":2402,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.563616+0000","last_change":"2024-09-15T21:18:17.166757+0000","last_active":"2024-09-15T21:18:21.563616+0000","last_peered":"2024-09-15T21:18:21.563616+0000","last_clean":"2024-09-15T21:18:21.563616+0000","last_became_active":"2024-09-15T21:17:45.443117+0000","last_became_peered":"2024-09-15T21:17:45.443117+0000","last_unstale":"2024-09-15T21:18:21.563616+0000","last_undegraded":"2024-09-15T21:18:21.563616+0000","last_fullsized":"2024-09-15T21:18:21.563616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2343,"log_dups_size":0,"ondisk_log_size":2343,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9565394,"num_objects":2337,"num_object_clones":0,"num_object_copies":4674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2341,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":71,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:17.499324+0000","last_change":"2024-09-15T21:18:17.499242+0000","last_active":"2024-09-15T21:18:17.499324+0000","last_peered":"2024-09-15T21:18:17.499324+0000","last_clean":"2024-09-15T21:18:17.499324+0000","last_became_active":"2024-09-15T21:17:45.437663+0000","last_became_peered":"2024-09-15T21:17:45.437663+0000","last_unstale":"2024-09-15T21:18:17.499324+0000","last_undegraded":"2024-09-15T21:18:17.499324+0000","last_fullsized":"2024-09-15T21:18:17.499324+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:43.848294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"37'2364","reported_seq":2448,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.394900+0000","last_change":"2024-09-15T21:18:09.166295+0000","last_active":"2024-09-15T21:18:23.394900+0000","last_peered":"2024-09-15T21:18:23.394900+0000","last_clean":"2024-09-15T21:18:23.394900+0000","last_became_active":"2024-09-15T21:17:46.444776+0000","last_became_peered":"2024-09-15T21:17:46.444776+0000","last_unstale":"2024-09-15T21:18:23.394900+0000","last_undegraded":"2024-09-15T21:18:23.394900+0000","last_fullsized":"2024-09-15T21:18:23.394900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:26.846753+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9662464,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":915,"num_write_kb":3660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.453116+0000","last_change":"2024-09-15T21:17:46.448474+0000","last_active":"2024-09-15T21:17:46.453116+0000","last_peered":"2024-09-15T21:17:46.453116+0000","last_clean":"2024-09-15T21:17:46.453116+0000","last_became_active":"2024-09-15T21:17:46.447324+0000","last_became_peered":"2024-09-15T21:17:46.447324+0000","last_unstale":"2024-09-15T21:17:46.453116+0000","last_undegraded":"2024-09-15T21:17:46.453116+0000","last_fullsized":"2024-09-15T21:17:46.453116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:02.995360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"37'2438","reported_seq":2525,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:23.362767+0000","last_change":"2024-09-15T21:18:10.504386+0000","last_active":"2024-09-15T21:18:23.362767+0000","last_peered":"2024-09-15T21:18:23.362767+0000","last_clean":"2024-09-15T21:18:23.362767+0000","last_became_active":"2024-09-15T21:17:46.444267+0000","last_became_peered":"2024-09-15T21:17:46.444267+0000","last_unstale":"2024-09-15T21:18:23.362767+0000","last_undegraded":"2024-09-15T21:18:23.362767+0000","last_fullsized":"2024-09-15T21:18:23.362767+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":2438,"log_dups_size":0,"ondisk_log_size":2438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.869337+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9961472,"num_objects":2432,"num_object_clones":0,"num_object_copies":4864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":837,"num_write_kb":3348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"37'2386","reported_seq":2450,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.563372+0000","last_change":"2024-09-15T21:18:17.489111+0000","last_active":"2024-09-15T21:18:21.563372+0000","last_peered":"2024-09-15T21:18:21.563372+0000","last_clean":"2024-09-15T21:18:21.563372+0000","last_became_active":"2024-09-15T21:17:45.443239+0000","last_became_peered":"2024-09-15T21:17:45.443239+0000","last_unstale":"2024-09-15T21:18:21.563372+0000","last_undegraded":"2024-09-15T21:18:21.563372+0000","last_fullsized":"2024-09-15T21:18:21.563372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2386,"log_dups_size":0,"ondisk_log_size":2386,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9765481,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2385,"num_write_kb":9537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.434157+0000","last_change":"2024-09-15T21:17:45.440637+0000","last_active":"2024-09-15T21:17:46.434157+0000","last_peered":"2024-09-15T21:17:46.434157+0000","last_clean":"2024-09-15T21:17:46.434157+0000","last_became_active":"2024-09-15T21:17:45.440347+0000","last_became_peered":"2024-09-15T21:17:45.440347+0000","last_unstale":"2024-09-15T21:17:46.434157+0000","last_undegraded":"2024-09-15T21:17:46.434157+0000","last_fullsized":"2024-09-15T21:17:46.434157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:54.708396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"37'2341","reported_seq":2410,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.563651+0000","last_change":"2024-09-15T21:18:17.100632+0000","last_active":"2024-09-15T21:18:21.563651+0000","last_peered":"2024-09-15T21:18:21.563651+0000","last_clean":"2024-09-15T21:18:21.563651+0000","last_became_active":"2024-09-15T21:17:45.443078+0000","last_became_peered":"2024-09-15T21:17:45.443078+0000","last_unstale":"2024-09-15T21:18:21.563651+0000","last_undegraded":"2024-09-15T21:18:21.563651+0000","last_fullsized":"2024-09-15T21:18:21.563651+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2341,"log_dups_size":0,"ondisk_log_size":2341,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9557819,"num_objects":2336,"num_object_clones":0,"num_object_copies":4672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.444413+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:17:46.444413+0000","last_peered":"2024-09-15T21:17:46.444413+0000","last_clean":"2024-09-15T21:17:46.444413+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:17:46.444413+0000","last_undegraded":"2024-09-15T21:17:46.444413+0000","last_fullsized":"2024-09-15T21:17:46.444413+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"37'2341","reported_seq":2406,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.563559+0000","last_change":"2024-09-15T21:18:17.010806+0000","last_active":"2024-09-15T21:18:21.563559+0000","last_peered":"2024-09-15T21:18:21.563559+0000","last_clean":"2024-09-15T21:18:21.563559+0000","last_became_active":"2024-09-15T21:17:45.443074+0000","last_became_peered":"2024-09-15T21:17:45.443074+0000","last_unstale":"2024-09-15T21:18:21.563559+0000","last_undegraded":"2024-09-15T21:18:21.563559+0000","last_fullsized":"2024-09-15T21:18:21.563559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":2341,"log_dups_size":0,"ondisk_log_size":2341,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9573586,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2339,"num_write_kb":9350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.433559+0000","last_change":"2024-09-15T21:17:45.440167+0000","last_active":"2024-09-15T21:17:46.433559+0000","last_peered":"2024-09-15T21:17:46.433559+0000","last_clean":"2024-09-15T21:17:46.433559+0000","last_became_active":"2024-09-15T21:17:45.439923+0000","last_became_peered":"2024-09-15T21:17:45.439923+0000","last_unstale":"2024-09-15T21:17:46.433559+0000","last_undegraded":"2024-09-15T21:17:46.433559+0000","last_fullsized":"2024-09-15T21:17:46.433559+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:59.972752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:17:46.448119+0000","last_change":"2024-09-15T21:17:46.445003+0000","last_active":"2024-09-15T21:17:46.448119+0000","last_peered":"2024-09-15T21:17:46.448119+0000","last_clean":"2024-09-15T21:17:46.448119+0000","last_became_active":"2024-09-15T21:17:46.444481+0000","last_became_peered":"2024-09-15T21:17:46.444481+0000","last_unstale":"2024-09-15T21:17:46.448119+0000","last_undegraded":"2024-09-15T21:17:46.448119+0000","last_fullsized":"2024-09-15T21:17:46.448119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:29.812368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"37'2434","reported_seq":2537,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:18:21.550712+0000","last_change":"2024-09-15T21:18:20.389833+0000","last_active":"2024-09-15T21:18:21.550712+0000","last_peered":"2024-09-15T21:18:21.550712+0000","last_clean":"2024-09-15T21:18:21.550712+0000","last_became_active":"2024-09-15T21:17:45.437293+0000","last_became_peered":"2024-09-15T21:17:45.437293+0000","last_unstale":"2024-09-15T21:18:21.550712+0000","last_undegraded":"2024-09-15T21:18:21.550712+0000","last_fullsized":"2024-09-15T21:18:21.550712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:43.173916+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9921129,"num_objects":2423,"num_object_clones":0,"num_object_copies":4846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156790613,"num_objects":38290,"num_object_clones":0,"num_object_copies":76580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24635,"num_write_kb":98474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315318272,"data_stored":315213902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38359,"ondisk_log_size":38359,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953485,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236476,"kb_used_data":154796,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94135364,"statfs":{"total":96636764160,"available":96394612736,"internally_reserved":0,"allocated":158511104,"data_stored":158289058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":83619501},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149713,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241108,"kb_used_data":147012,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94130732,"statfs":{"total":96636764160,"available":96389869568,"internally_reserved":0,"allocated":150540288,"data_stored":150318242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":96333489},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":154824704,"data_stored":154821225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":160378880,"data_stored":160375401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:25.790 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-15T21:18:25.790 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-15T21:18:25.790 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-15T21:18:26.080 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-15T21:18:26.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.085 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-15T21:18:26.085 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-15T21:18:26.204 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-15T21:18:26.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.209 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-15T21:18:26.209 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-15T21:18:26.333 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-15T21:18:26.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.342 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-15T21:18:26.342 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-15T21:18:26.430 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-15T21:18:26.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.436 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-15T21:18:26.436 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-15T21:18:26.472 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 33 20 42742 42722 5.05654 4.64453 0.0213888 0.0154426 2024-09-15T21:18:26.542 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-15T21:18:26.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.548 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-15T21:18:26.548 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-15T21:18:26.700 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-15T21:18:26.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.709 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-15T21:18:26.709 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-15T21:18:26.794 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-15T21:18:26.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.812 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-15T21:18:26.812 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-15T21:18:26.879 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-15T21:18:26.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.890 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-15T21:18:26.890 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-15T21:18:26.961 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-15T21:18:26.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:26.965 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-15T21:18:26.965 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-15T21:18:27.045 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-15T21:18:27.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.059 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-15T21:18:27.059 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-15T21:18:27.141 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-15T21:18:27.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.162 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-15T21:18:27.162 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-15T21:18:27.256 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-15T21:18:27.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.263 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-15T21:18:27.263 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-15T21:18:27.339 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-15T21:18:27.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.348 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-15T21:18:27.349 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-15T21:18:27.414 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-15T21:18:27.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.428 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-15T21:18:27.428 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-15T21:18:27.472 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 34 20 43990 43970 5.05119 4.875 0.00754529 0.0154614 2024-09-15T21:18:27.499 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-15T21:18:27.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.508 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-15T21:18:27.508 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-15T21:18:27.639 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-15T21:18:27.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.642 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-15T21:18:27.643 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-15T21:18:27.676 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-15T21:18:27.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.679 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-15T21:18:27.680 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-15T21:18:27.755 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-15T21:18:27.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.758 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-15T21:18:27.758 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-15T21:18:27.837 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-15T21:18:27.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.847 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-15T21:18:27.847 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-15T21:18:27.965 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-15T21:18:27.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:27.977 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-15T21:18:27.978 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-15T21:18:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-15T21:18:28.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.032 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-15T21:18:28.032 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-15T21:18:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-15T21:18:28.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.069 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-15T21:18:28.069 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-15T21:18:28.147 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-15T21:18:28.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.150 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-15T21:18:28.150 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-15T21:18:28.453 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-15T21:18:28.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.465 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-15T21:18:28.465 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-15T21:18:28.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 35 20 44799 44779 4.99703 3.16016 0.00622813 0.0156265 2024-09-15T21:18:28.541 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-15T21:18:28.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.556 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-15T21:18:28.557 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-15T21:18:28.706 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-15T21:18:28.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.713 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-15T21:18:28.713 DEBUG:teuthology.orchestra.run.smithi114:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-15T21:18:28.822 INFO:teuthology.orchestra.run.smithi114.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-15T21:18:28.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:18:28.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-15T21:18:29.196 INFO:tasks.ceph.mon.a.smithi114.stderr:2024-09-15T21:18:29.191+0000 7fefb254a640 -1 mon.a@0(leader).osd e37 definitely_dead 0 2024-09-15T21:18:29.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 36 20 45852 45832 4.97248 4.11328 0.0113847 0.0157046 2024-09-15T21:18:29.649 INFO:teuthology.orchestra.run.smithi114.stderr:marked in osd.1. 2024-09-15T21:18:29.681 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:18:29.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:18:29.683 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:18:29.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-15T21:18:29.782 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-15T21:18:29.783 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-15T21:18:29.783 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:18:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:18:29.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T21:18:30.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 37 20 46121 46101 4.86648 1.05078 0.00315591 0.0156994 2024-09-15T21:18:30.763 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:18:30.759+0000 7fa3d20f07c0 -1 Falling back to public interface 2024-09-15T21:18:31.473 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 38 20 46121 46101 4.73841 0 - 0.0156994 2024-09-15T21:18:32.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 39 20 46818 46798 4.68653 1.36133 0.00724658 0.0166647 2024-09-15T21:18:33.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:18:33.474226+0000 min lat: 0.00128075 max lat: 2.08264 avg lat: 0.0168068 2024-09-15T21:18:33.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:18:33.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 40 20 47606 47586 4.64632 3.07812 0.00556835 0.0168068 2024-09-15T21:18:34.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 41 20 47884 47864 4.55948 1.08594 0.00302671 0.0169753 2024-09-15T21:18:34.785 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:18:35.014 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:18:35.014 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:18:35.014 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:18:35.014 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-15T21:18:35.015 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:18:35.016 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:18:35.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T21:18:35.475 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 42 20 48785 48765 4.53471 3.51953 0.0182623 0.0172215 2024-09-15T21:18:35.647 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:18:35.643+0000 7fa3d20f07c0 -1 osd.1 27 log_to_monitors true 2024-09-15T21:18:36.476 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 43 20 49780 49760 4.5196 3.88672 0.0091178 0.0172759 2024-09-15T21:18:37.476 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 44 20 50859 50839 4.51267 4.21484 0.0190596 0.0173054 2024-09-15T21:18:38.477 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 45 20 51717 51697 4.48677 3.35156 0.0151756 0.0174071 2024-09-15T21:18:39.477 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 46 20 52593 52573 4.46362 3.42188 0.0082142 0.017496 2024-09-15T21:18:40.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:18:40.283 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs files list 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs stats 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator dump block 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation block 2024-09-15T21:18:40.284 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-15T21:18:40.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:18:40.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T21:18:40.348 INFO:tasks.ceph.osd.1.smithi114.stderr:2024-09-15T21:18:40.339+0000 7fa3c5ecb640 -1 osd.1 27 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:18:40.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 47 20 53695 53675 4.4602 4.30469 0.0115744 0.0175103 2024-09-15T21:18:41.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 48 20 54286 54266 4.41537 2.30859 0.00252075 0.0176451 2024-09-15T21:18:42.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 49 20 54286 54266 4.32526 0 - 0.0176451 2024-09-15T21:18:43.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 50 20 54286 54266 4.23876 0 - 0.0176451 2024-09-15T21:18:44.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 51 20 54594 54574 4.17924 0.401042 0.0189159 0.0186247 2024-09-15T21:18:45.288 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:18:45.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 52 20 55274 55254 4.14994 2.65625 0.0508116 0.0188135 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55318 2.d e44/43 2:b00ec157:::benchmark_data_smithi114_33359_object55317:head v 44'3462, mlcod=37'58)", 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.564446+0000", 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.018347472, 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018369778, 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.585 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564446+0000", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564446+0000", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564447+0000", 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.083e-06 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564469+0000", 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1503999999999999e-05 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564482+0000", 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.385e-05 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.564490+0000", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2420000000000003e-06 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572419+0000", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079297169999999993 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572439+0000", 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9385000000000002e-05 2024-09-15T21:18:45.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.582693+0000", 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010254386000000001 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55324 2.a e44/43 2:57b6c965:::benchmark_data_smithi114_33359_object55323:head v 44'3450, mlcod=24'3)", 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.568436+0000", 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.014357958000000001, 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014438879999999999, 2024-09-15T21:18:45.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568436+0000", 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568436+0000", 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568437+0000", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9309999999999998e-06 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568449+0000", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1308000000000001e-05 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568460+0000", 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0896e-05 2024-09-15T21:18:45.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.568468+0000", 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8449999999999992e-06 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571594+0000", 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003125526 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571624+0000", 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9799999999999999e-05 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55328 2.9 e44/43 2:92c27398:::benchmark_data_smithi114_33359_object55327:head v 44'3559, mlcod=24'4)", 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.571823+0000", 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.010970288999999999, 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011102182, 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-15T21:18:45.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571823+0000", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571823+0000", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571824+0000", 2024-09-15T21:18:45.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.198e-06 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571834+0000", 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9879999999999999e-06 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571845+0000", 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0745e-05 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571853+0000", 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7249999999999997e-06 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571902+0000", 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8878000000000002e-05 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571919+0000", 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7476999999999999e-05 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55327 2.4 e44/43 2:2fe1ca83:::benchmark_data_smithi114_33359_object55326:head v 44'3551, mlcod=24'9)", 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.571872+0000", 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.010921322000000001, 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011098395, 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571872+0000", 2024-09-15T21:18:45.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571872+0000", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571872+0000", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571882+0000", 2024-09-15T21:18:45.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0189999999999995e-06 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571887+0000", 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.643e-06 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571892+0000", 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5719999999999996e-06 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571943+0000", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0997e-05 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.571965+0000", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2476999999999998e-05 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.582683+0000", 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01071796 2024-09-15T21:18:45.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55333 2.5 e44/43 2:ae2f13bd:::benchmark_data_smithi114_33359_object55332:head v 44'3430, mlcod=24'2)", 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.572706+0000", 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.010087148000000001, 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010316854, 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572706+0000", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572706+0000", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572707+0000", 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572723+0000", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6005999999999999e-05 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572731+0000", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1699999999999997e-06 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.572738+0000", 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2079999999999997e-06 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.576282+0000", 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003544171 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.576310+0000", 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8141e-05 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55335 2.9 e44/43 2:9ec1a32a:::benchmark_data_smithi114_33359_object55334:head v 44'3560, mlcod=24'4)", 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.578850+0000", 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0039437999999999999, 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042192660000000002, 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578850+0000", 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578850+0000", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578851+0000", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8200000000000008e-07 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578861+0000", 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9380000000000004e-06 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578874+0000", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3895e-05 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578882+0000", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578975+0000", 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2251999999999997e-05 2024-09-15T21:18:45.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.578998+0000", 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3323999999999999e-05 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.5110.0:55334 2.f e44/43 2:fbb0bb5c:::benchmark_data_smithi114_33359_object55333:head v 44'3423, mlcod=0'0)", 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-15T21:18:45.581100+0000", 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0016933810000000001, 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020153039999999999, 2024-09-15T21:18:45.608 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581100+0000", 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581100+0000", 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581101+0000", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3600000000000002e-07 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581111+0000", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0342e-05 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581126+0000", 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4674e-05 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581135+0000", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7260000000000008e-06 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581227+0000", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2093000000000001e-05 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-15T21:18:45.581247+0000", 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0718000000000001e-05 2024-09-15T21:18:45.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 7 2024-09-15T21:18:45.612 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:18:45.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-15T21:18:45.626 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:18:46.303 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:46.304 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":44,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:18:43.355329+0000","last_up_change":"2024-09-15T21:18:41.342812+0000","last_in_change":"2024-09-15T21:18:29.636613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-15T21:16:54.321157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":43,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":490656443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":490656443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":490656443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":490656443}]},"public_addr":"172.21.15.114:6804/490656443","cluster_addr":"172.21.15.114:6805/490656443","heartbeat_back_addr":"172.21.15.114:6807/490656443","heartbeat_front_addr":"172.21.15.114:6806/490656443","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":27,"up_from":42,"up_thru":43,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1392950530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":1392950530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":1392950530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1392950530}]},"public_addr":"172.21.15.114:6808/1392950530","cluster_addr":"172.21.15.114:6809/1392950530","heartbeat_back_addr":"172.21.15.114:6811/1392950530","heartbeat_front_addr":"172.21.15.114:6810/1392950530","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":19,"up_from":32,"up_thru":43,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":42043953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":42043953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":42043953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":42043953}]},"public_addr":"172.21.15.114:6800/42043953","cluster_addr":"172.21.15.114:6801/42043953","heartbeat_back_addr":"172.21.15.114:6803/42043953","heartbeat_front_addr":"172.21.15.114:6802/42043953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:18:46.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-15T21:18:46.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-15T21:18:46.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:46.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 53 20 55813 55793 4.11136 2.10547 0.0864727 0.0189913 2024-09-15T21:18:46.998 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:47.047 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-15T21:18:46.306771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222315132,"num_objects":54279,"num_object_clones":0,"num_object_copies":108558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":54279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25774,"num_write_kb":103090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54361,"ondisk_log_size":54361,"up":46,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":676972,"kb_used_data":433392,"kb_used_omap":12,"kb_used_meta":243507,"kb_avail":282438548,"statfs":{"total":289910292480,"available":289217073152,"internally_reserved":0,"allocated":443793408,"data_stored":442967586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":249351178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":-37813,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3601,"num_write_kb":14404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001397"},"pg_stats":[{"pgid":"2.f","version":"41'3367","reported_seq":3431,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.352229+0000","last_change":"2024-09-15T21:18:41.364086+0000","last_active":"2024-09-15T21:18:41.364086+0000","last_peered":"2024-09-15T21:18:41.352229+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:31.699774+0000","last_became_peered":"2024-09-15T21:18:31.699774+0000","last_unstale":"2024-09-15T21:18:41.352229+0000","last_undegraded":"2024-09-15T21:18:41.352229+0000","last_fullsized":"2024-09-15T21:18:41.352229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3367,"log_dups_size":0,"ondisk_log_size":3367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3367,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3419","reported_seq":3510,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371973+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:43.371973+0000","last_peered":"2024-09-15T21:18:43.371973+0000","last_clean":"2024-09-15T21:18:43.371973+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:43.371973+0000","last_undegraded":"2024-09-15T21:18:43.371973+0000","last_fullsized":"2024-09-15T21:18:43.371973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3419,"log_dups_size":0,"ondisk_log_size":3419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3396","reported_seq":3507,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.372668+0000","last_change":"2024-09-15T21:18:43.436365+0000","last_active":"2024-09-15T21:18:41.362222+0000","last_peered":"2024-09-15T21:18:41.353008+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:31.680292+0000","last_became_peered":"2024-09-15T21:18:31.680292+0000","last_unstale":"2024-09-15T21:18:43.372668+0000","last_undegraded":"2024-09-15T21:18:43.372668+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3396,"log_dups_size":0,"ondisk_log_size":3396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893632,"num_objects":3392,"num_object_clones":0,"num_object_copies":6784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2119,"num_write_kb":8476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'3462","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371797+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:43.371797+0000","last_peered":"2024-09-15T21:18:43.371797+0000","last_clean":"2024-09-15T21:18:43.371797+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:43.371797+0000","last_undegraded":"2024-09-15T21:18:43.371797+0000","last_fullsized":"2024-09-15T21:18:43.371797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3462,"log_dups_size":0,"ondisk_log_size":3462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14180352,"num_objects":3462,"num_object_clones":0,"num_object_copies":6924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"42'3421","reported_seq":3530,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T21:18:41.357172+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:41.357172+0000","last_peered":"2024-09-15T21:18:41.357172+0000","last_clean":"2024-09-15T21:18:41.357172+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:41.357172+0000","last_undegraded":"2024-09-15T21:18:41.357172+0000","last_fullsized":"2024-09-15T21:18:41.357172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3421,"log_dups_size":0,"ondisk_log_size":3421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13983744,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":5116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'3376","reported_seq":3492,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.310391+0000","last_change":"2024-09-15T21:18:41.354793+0000","last_active":"2024-09-15T21:18:41.354793+0000","last_peered":"2024-09-15T21:18:41.310391+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:31.675833+0000","last_became_peered":"2024-09-15T21:18:31.675833+0000","last_unstale":"2024-09-15T21:18:41.310391+0000","last_undegraded":"2024-09-15T21:18:41.310391+0000","last_fullsized":"2024-09-15T21:18:41.310391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13807616,"num_objects":3371,"num_object_clones":0,"num_object_copies":6742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":3204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'3477","reported_seq":3587,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.345061+0000","last_change":"2024-09-15T21:18:41.368795+0000","last_active":"2024-09-15T21:18:41.368795+0000","last_peered":"2024-09-15T21:18:41.345061+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:31.676092+0000","last_became_peered":"2024-09-15T21:18:31.676092+0000","last_unstale":"2024-09-15T21:18:41.345061+0000","last_undegraded":"2024-09-15T21:18:41.345061+0000","last_fullsized":"2024-09-15T21:18:41.345061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3477,"log_dups_size":0,"ondisk_log_size":3477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14217216,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1113,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"42'3446","reported_seq":3556,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T21:18:41.378790+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:41.378790+0000","last_peered":"2024-09-15T21:18:41.378790+0000","last_clean":"2024-09-15T21:18:41.378790+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:41.378790+0000","last_undegraded":"2024-09-15T21:18:41.378790+0000","last_fullsized":"2024-09-15T21:18:41.378790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3446,"log_dups_size":0,"ondisk_log_size":3446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14106624,"num_objects":3444,"num_object_clones":0,"num_object_copies":6888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.375219+0000","last_change":"2024-09-15T21:18:41.355603+0000","last_active":"2024-09-15T21:18:41.355603+0000","last_peered":"2024-09-15T21:18:36.375219+0000","last_clean":"2024-09-15T21:18:36.375219+0000","last_became_active":"2024-09-15T21:18:31.680739+0000","last_became_peered":"2024-09-15T21:18:31.680739+0000","last_unstale":"2024-09-15T21:18:36.375219+0000","last_undegraded":"2024-09-15T21:18:36.375219+0000","last_fullsized":"2024-09-15T21:18:36.375219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3345","reported_seq":3460,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369078+0000","last_change":"2024-09-15T21:18:43.442399+0000","last_active":"2024-09-15T21:18:41.362286+0000","last_peered":"2024-09-15T21:18:41.351933+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:31.682424+0000","last_became_peered":"2024-09-15T21:18:31.682424+0000","last_unstale":"2024-09-15T21:18:43.369078+0000","last_undegraded":"2024-09-15T21:18:43.369078+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3345,"log_dups_size":0,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'3349","reported_seq":3458,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.348908+0000","last_change":"2024-09-15T21:18:41.356235+0000","last_active":"2024-09-15T21:18:41.356235+0000","last_peered":"2024-09-15T21:18:41.348908+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:31.681701+0000","last_became_peered":"2024-09-15T21:18:31.681701+0000","last_unstale":"2024-09-15T21:18:41.348908+0000","last_undegraded":"2024-09-15T21:18:41.348908+0000","last_fullsized":"2024-09-15T21:18:41.348908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3349,"log_dups_size":0,"ondisk_log_size":3349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13684736,"num_objects":3341,"num_object_clones":0,"num_object_copies":6682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":85,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376550+0000","last_change":"2024-09-15T21:18:41.355015+0000","last_active":"2024-09-15T21:18:41.355015+0000","last_peered":"2024-09-15T21:18:36.376550+0000","last_clean":"2024-09-15T21:18:36.376550+0000","last_became_active":"2024-09-15T21:18:31.675074+0000","last_became_peered":"2024-09-15T21:18:31.675074+0000","last_unstale":"2024-09-15T21:18:36.376550+0000","last_undegraded":"2024-09-15T21:18:36.376550+0000","last_fullsized":"2024-09-15T21:18:36.376550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3271","reported_seq":3371,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369147+0000","last_change":"2024-09-15T21:18:43.442812+0000","last_active":"2024-09-15T21:18:41.354478+0000","last_peered":"2024-09-15T21:18:41.345310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:31.708127+0000","last_became_peered":"2024-09-15T21:18:31.708127+0000","last_unstale":"2024-09-15T21:18:43.369147+0000","last_undegraded":"2024-09-15T21:18:43.369147+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3271,"log_dups_size":0,"ondisk_log_size":3271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13377536,"num_objects":3266,"num_object_clones":0,"num_object_copies":6532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3266}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":29,"reported_epoch":41,"state":"active+clean+remapped","last_fresh":"2024-09-15T21:18:36.377115+0000","last_change":"2024-09-15T21:18:31.683499+0000","last_active":"2024-09-15T21:18:36.377115+0000","last_peered":"2024-09-15T21:18:36.377115+0000","last_clean":"2024-09-15T21:18:36.377115+0000","last_became_active":"2024-09-15T21:18:31.682720+0000","last_became_peered":"2024-09-15T21:18:31.682720+0000","last_unstale":"2024-09-15T21:18:36.377115+0000","last_undegraded":"2024-09-15T21:18:36.377115+0000","last_fullsized":"2024-09-15T21:18:36.377115+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:24.235578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3356","reported_seq":3461,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.368584+0000","last_change":"2024-09-15T21:18:43.448670+0000","last_active":"2024-09-15T21:18:41.357225+0000","last_peered":"2024-09-15T21:18:41.344494+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:31.696760+0000","last_became_peered":"2024-09-15T21:18:31.696760+0000","last_unstale":"2024-09-15T21:18:43.368584+0000","last_undegraded":"2024-09-15T21:18:43.368584+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3356,"log_dups_size":0,"ondisk_log_size":3356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13721600,"num_objects":3350,"num_object_clones":0,"num_object_copies":6700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":7020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'3435","reported_seq":3550,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.355508+0000","last_change":"2024-09-15T21:18:41.366757+0000","last_active":"2024-09-15T21:18:41.366757+0000","last_peered":"2024-09-15T21:18:41.355508+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:31.697658+0000","last_became_peered":"2024-09-15T21:18:31.697658+0000","last_unstale":"2024-09-15T21:18:41.355508+0000","last_undegraded":"2024-09-15T21:18:41.355508+0000","last_fullsized":"2024-09-15T21:18:41.355508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3435,"log_dups_size":0,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14061568,"num_objects":3433,"num_object_clones":0,"num_object_copies":6866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":742,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376129+0000","last_change":"2024-09-15T21:18:41.354475+0000","last_active":"2024-09-15T21:18:41.354475+0000","last_peered":"2024-09-15T21:18:36.376129+0000","last_clean":"2024-09-15T21:18:36.376129+0000","last_became_active":"2024-09-15T21:18:31.685912+0000","last_became_peered":"2024-09-15T21:18:31.685912+0000","last_unstale":"2024-09-15T21:18:36.376129+0000","last_undegraded":"2024-09-15T21:18:36.376129+0000","last_fullsized":"2024-09-15T21:18:36.376129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"41'3387","reported_seq":3503,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.339426+0000","last_change":"2024-09-15T21:18:41.366527+0000","last_active":"2024-09-15T21:18:41.366527+0000","last_peered":"2024-09-15T21:18:41.339426+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:31.676412+0000","last_became_peered":"2024-09-15T21:18:31.676412+0000","last_unstale":"2024-09-15T21:18:41.339426+0000","last_undegraded":"2024-09-15T21:18:41.339426+0000","last_fullsized":"2024-09-15T21:18:41.339426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3387,"log_dups_size":0,"ondisk_log_size":3387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13840384,"num_objects":3379,"num_object_clones":0,"num_object_copies":6758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'3369","reported_seq":3479,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.339576+0000","last_change":"2024-09-15T21:18:41.356377+0000","last_active":"2024-09-15T21:18:41.356377+0000","last_peered":"2024-09-15T21:18:41.339576+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:31.675789+0000","last_became_peered":"2024-09-15T21:18:31.675789+0000","last_unstale":"2024-09-15T21:18:41.339576+0000","last_undegraded":"2024-09-15T21:18:41.339576+0000","last_fullsized":"2024-09-15T21:18:41.339576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3369,"log_dups_size":0,"ondisk_log_size":3369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13783040,"num_objects":3365,"num_object_clones":0,"num_object_copies":6730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376338+0000","last_change":"2024-09-15T21:18:41.356794+0000","last_active":"2024-09-15T21:18:41.356794+0000","last_peered":"2024-09-15T21:18:36.376338+0000","last_clean":"2024-09-15T21:18:36.376338+0000","last_became_active":"2024-09-15T21:18:31.675622+0000","last_became_peered":"2024-09-15T21:18:31.675622+0000","last_unstale":"2024-09-15T21:18:36.376338+0000","last_undegraded":"2024-09-15T21:18:36.376338+0000","last_fullsized":"2024-09-15T21:18:36.376338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":41,"state":"active+clean+remapped","last_fresh":"2024-09-15T21:18:36.377709+0000","last_change":"2024-09-15T21:18:31.716253+0000","last_active":"2024-09-15T21:18:36.377709+0000","last_peered":"2024-09-15T21:18:36.377709+0000","last_clean":"2024-09-15T21:18:36.377709+0000","last_became_active":"2024-09-15T21:18:31.713938+0000","last_became_peered":"2024-09-15T21:18:31.713938+0000","last_unstale":"2024-09-15T21:18:36.377709+0000","last_undegraded":"2024-09-15T21:18:36.377709+0000","last_fullsized":"2024-09-15T21:18:36.377709+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:20.500541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"41'3482","reported_seq":3597,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.321065+0000","last_change":"2024-09-15T21:18:41.355200+0000","last_active":"2024-09-15T21:18:41.355200+0000","last_peered":"2024-09-15T21:18:41.321065+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:31.676228+0000","last_became_peered":"2024-09-15T21:18:31.676228+0000","last_unstale":"2024-09-15T21:18:41.321065+0000","last_undegraded":"2024-09-15T21:18:41.321065+0000","last_fullsized":"2024-09-15T21:18:41.321065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3482,"log_dups_size":0,"ondisk_log_size":3482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14213737,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1142,"num_write_kb":4568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222315113,"num_objects":54277,"num_object_clones":0,"num_object_copies":108554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25772,"num_write_kb":103088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":444723200,"data_stored":444636225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54358,"ondisk_log_size":54358,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325512,"kb_used_data":217456,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94046328,"statfs":{"total":96636764160,"available":96303439872,"internally_reserved":0,"allocated":222674944,"data_stored":222424661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":110620333},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149717,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319308,"kb_used_data":215484,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94052532,"statfs":{"total":96636764160,"available":96309792768,"internally_reserved":0,"allocated":220655616,"data_stored":220414813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":106294961},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":462848,"data_stored":128112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":32435884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222318592,"data_stored":222315113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":14191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222310400,"data_stored":222306921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:47.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:47.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 54 20 56339 56319 4.07328 2.05469 0.0106441 0.0191691 2024-09-15T21:18:47.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:47.546 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:47.610 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-15T21:18:46.306771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222315132,"num_objects":54279,"num_object_clones":0,"num_object_copies":108558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":54279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25774,"num_write_kb":103090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54361,"ondisk_log_size":54361,"up":46,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":676972,"kb_used_data":433392,"kb_used_omap":12,"kb_used_meta":243507,"kb_avail":282438548,"statfs":{"total":289910292480,"available":289217073152,"internally_reserved":0,"allocated":443793408,"data_stored":442967586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":249351178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":-37813,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3601,"num_write_kb":14404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001397"},"pg_stats":[{"pgid":"2.f","version":"41'3367","reported_seq":3431,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.352229+0000","last_change":"2024-09-15T21:18:41.364086+0000","last_active":"2024-09-15T21:18:41.364086+0000","last_peered":"2024-09-15T21:18:41.352229+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:31.699774+0000","last_became_peered":"2024-09-15T21:18:31.699774+0000","last_unstale":"2024-09-15T21:18:41.352229+0000","last_undegraded":"2024-09-15T21:18:41.352229+0000","last_fullsized":"2024-09-15T21:18:41.352229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3367,"log_dups_size":0,"ondisk_log_size":3367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3367,"num_write_kb":13468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3419","reported_seq":3510,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371973+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:43.371973+0000","last_peered":"2024-09-15T21:18:43.371973+0000","last_clean":"2024-09-15T21:18:43.371973+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:43.371973+0000","last_undegraded":"2024-09-15T21:18:43.371973+0000","last_fullsized":"2024-09-15T21:18:43.371973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3419,"log_dups_size":0,"ondisk_log_size":3419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3396","reported_seq":3507,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.372668+0000","last_change":"2024-09-15T21:18:43.436365+0000","last_active":"2024-09-15T21:18:41.362222+0000","last_peered":"2024-09-15T21:18:41.353008+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:31.680292+0000","last_became_peered":"2024-09-15T21:18:31.680292+0000","last_unstale":"2024-09-15T21:18:43.372668+0000","last_undegraded":"2024-09-15T21:18:43.372668+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3396,"log_dups_size":0,"ondisk_log_size":3396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893632,"num_objects":3392,"num_object_clones":0,"num_object_copies":6784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2119,"num_write_kb":8476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'3462","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371797+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:43.371797+0000","last_peered":"2024-09-15T21:18:43.371797+0000","last_clean":"2024-09-15T21:18:43.371797+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:43.371797+0000","last_undegraded":"2024-09-15T21:18:43.371797+0000","last_fullsized":"2024-09-15T21:18:43.371797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3462,"log_dups_size":0,"ondisk_log_size":3462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14180352,"num_objects":3462,"num_object_clones":0,"num_object_copies":6924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"42'3421","reported_seq":3530,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T21:18:41.357172+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:41.357172+0000","last_peered":"2024-09-15T21:18:41.357172+0000","last_clean":"2024-09-15T21:18:41.357172+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:41.357172+0000","last_undegraded":"2024-09-15T21:18:41.357172+0000","last_fullsized":"2024-09-15T21:18:41.357172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3421,"log_dups_size":0,"ondisk_log_size":3421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13983744,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1279,"num_write_kb":5116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'3376","reported_seq":3492,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.310391+0000","last_change":"2024-09-15T21:18:41.354793+0000","last_active":"2024-09-15T21:18:41.354793+0000","last_peered":"2024-09-15T21:18:41.310391+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:31.675833+0000","last_became_peered":"2024-09-15T21:18:31.675833+0000","last_unstale":"2024-09-15T21:18:41.310391+0000","last_undegraded":"2024-09-15T21:18:41.310391+0000","last_fullsized":"2024-09-15T21:18:41.310391+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13807616,"num_objects":3371,"num_object_clones":0,"num_object_copies":6742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":801,"num_write_kb":3204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'3477","reported_seq":3587,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.345061+0000","last_change":"2024-09-15T21:18:41.368795+0000","last_active":"2024-09-15T21:18:41.368795+0000","last_peered":"2024-09-15T21:18:41.345061+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:31.676092+0000","last_became_peered":"2024-09-15T21:18:31.676092+0000","last_unstale":"2024-09-15T21:18:41.345061+0000","last_undegraded":"2024-09-15T21:18:41.345061+0000","last_fullsized":"2024-09-15T21:18:41.345061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3477,"log_dups_size":0,"ondisk_log_size":3477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14217216,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1113,"num_write_kb":4452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"42'3446","reported_seq":3556,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-15T21:18:41.378790+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:41.378790+0000","last_peered":"2024-09-15T21:18:41.378790+0000","last_clean":"2024-09-15T21:18:41.378790+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:41.378790+0000","last_undegraded":"2024-09-15T21:18:41.378790+0000","last_fullsized":"2024-09-15T21:18:41.378790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3446,"log_dups_size":0,"ondisk_log_size":3446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14106624,"num_objects":3444,"num_object_clones":0,"num_object_copies":6888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.375219+0000","last_change":"2024-09-15T21:18:41.355603+0000","last_active":"2024-09-15T21:18:41.355603+0000","last_peered":"2024-09-15T21:18:36.375219+0000","last_clean":"2024-09-15T21:18:36.375219+0000","last_became_active":"2024-09-15T21:18:31.680739+0000","last_became_peered":"2024-09-15T21:18:31.680739+0000","last_unstale":"2024-09-15T21:18:36.375219+0000","last_undegraded":"2024-09-15T21:18:36.375219+0000","last_fullsized":"2024-09-15T21:18:36.375219+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"41'3345","reported_seq":3460,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369078+0000","last_change":"2024-09-15T21:18:43.442399+0000","last_active":"2024-09-15T21:18:41.362286+0000","last_peered":"2024-09-15T21:18:41.351933+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:31.682424+0000","last_became_peered":"2024-09-15T21:18:31.682424+0000","last_unstale":"2024-09-15T21:18:43.369078+0000","last_undegraded":"2024-09-15T21:18:43.369078+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3345,"log_dups_size":0,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'3349","reported_seq":3458,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.348908+0000","last_change":"2024-09-15T21:18:41.356235+0000","last_active":"2024-09-15T21:18:41.356235+0000","last_peered":"2024-09-15T21:18:41.348908+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:31.681701+0000","last_became_peered":"2024-09-15T21:18:31.681701+0000","last_unstale":"2024-09-15T21:18:41.348908+0000","last_undegraded":"2024-09-15T21:18:41.348908+0000","last_fullsized":"2024-09-15T21:18:41.348908+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3349,"log_dups_size":0,"ondisk_log_size":3349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13684736,"num_objects":3341,"num_object_clones":0,"num_object_copies":6682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":875,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":85,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376550+0000","last_change":"2024-09-15T21:18:41.355015+0000","last_active":"2024-09-15T21:18:41.355015+0000","last_peered":"2024-09-15T21:18:36.376550+0000","last_clean":"2024-09-15T21:18:36.376550+0000","last_became_active":"2024-09-15T21:18:31.675074+0000","last_became_peered":"2024-09-15T21:18:31.675074+0000","last_unstale":"2024-09-15T21:18:36.376550+0000","last_undegraded":"2024-09-15T21:18:36.376550+0000","last_fullsized":"2024-09-15T21:18:36.376550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3271","reported_seq":3371,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369147+0000","last_change":"2024-09-15T21:18:43.442812+0000","last_active":"2024-09-15T21:18:41.354478+0000","last_peered":"2024-09-15T21:18:41.345310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:31.708127+0000","last_became_peered":"2024-09-15T21:18:31.708127+0000","last_unstale":"2024-09-15T21:18:43.369147+0000","last_undegraded":"2024-09-15T21:18:43.369147+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3271,"log_dups_size":0,"ondisk_log_size":3271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13377536,"num_objects":3266,"num_object_clones":0,"num_object_copies":6532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3266}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":29,"reported_epoch":41,"state":"active+clean+remapped","last_fresh":"2024-09-15T21:18:36.377115+0000","last_change":"2024-09-15T21:18:31.683499+0000","last_active":"2024-09-15T21:18:36.377115+0000","last_peered":"2024-09-15T21:18:36.377115+0000","last_clean":"2024-09-15T21:18:36.377115+0000","last_became_active":"2024-09-15T21:18:31.682720+0000","last_became_peered":"2024-09-15T21:18:31.682720+0000","last_unstale":"2024-09-15T21:18:36.377115+0000","last_undegraded":"2024-09-15T21:18:36.377115+0000","last_fullsized":"2024-09-15T21:18:36.377115+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:24.235578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"41'3356","reported_seq":3461,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.368584+0000","last_change":"2024-09-15T21:18:43.448670+0000","last_active":"2024-09-15T21:18:41.357225+0000","last_peered":"2024-09-15T21:18:41.344494+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:31.696760+0000","last_became_peered":"2024-09-15T21:18:31.696760+0000","last_unstale":"2024-09-15T21:18:43.368584+0000","last_undegraded":"2024-09-15T21:18:43.368584+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3356,"log_dups_size":0,"ondisk_log_size":3356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13721600,"num_objects":3350,"num_object_clones":0,"num_object_copies":6700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":7020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'3435","reported_seq":3550,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.355508+0000","last_change":"2024-09-15T21:18:41.366757+0000","last_active":"2024-09-15T21:18:41.366757+0000","last_peered":"2024-09-15T21:18:41.355508+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:31.697658+0000","last_became_peered":"2024-09-15T21:18:31.697658+0000","last_unstale":"2024-09-15T21:18:41.355508+0000","last_undegraded":"2024-09-15T21:18:41.355508+0000","last_fullsized":"2024-09-15T21:18:41.355508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3435,"log_dups_size":0,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14061568,"num_objects":3433,"num_object_clones":0,"num_object_copies":6866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":742,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376129+0000","last_change":"2024-09-15T21:18:41.354475+0000","last_active":"2024-09-15T21:18:41.354475+0000","last_peered":"2024-09-15T21:18:36.376129+0000","last_clean":"2024-09-15T21:18:36.376129+0000","last_became_active":"2024-09-15T21:18:31.685912+0000","last_became_peered":"2024-09-15T21:18:31.685912+0000","last_unstale":"2024-09-15T21:18:36.376129+0000","last_undegraded":"2024-09-15T21:18:36.376129+0000","last_fullsized":"2024-09-15T21:18:36.376129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"41'3387","reported_seq":3503,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.339426+0000","last_change":"2024-09-15T21:18:41.366527+0000","last_active":"2024-09-15T21:18:41.366527+0000","last_peered":"2024-09-15T21:18:41.339426+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:31.676412+0000","last_became_peered":"2024-09-15T21:18:31.676412+0000","last_unstale":"2024-09-15T21:18:41.339426+0000","last_undegraded":"2024-09-15T21:18:41.339426+0000","last_fullsized":"2024-09-15T21:18:41.339426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3387,"log_dups_size":0,"ondisk_log_size":3387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13840384,"num_objects":3379,"num_object_clones":0,"num_object_copies":6758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":972,"num_write_kb":3888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'3369","reported_seq":3479,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.339576+0000","last_change":"2024-09-15T21:18:41.356377+0000","last_active":"2024-09-15T21:18:41.356377+0000","last_peered":"2024-09-15T21:18:41.339576+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:31.675789+0000","last_became_peered":"2024-09-15T21:18:31.675789+0000","last_unstale":"2024-09-15T21:18:41.339576+0000","last_undegraded":"2024-09-15T21:18:41.339576+0000","last_fullsized":"2024-09-15T21:18:41.339576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3369,"log_dups_size":0,"ondisk_log_size":3369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13783040,"num_objects":3365,"num_object_clones":0,"num_object_copies":6730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":4064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:36.376338+0000","last_change":"2024-09-15T21:18:41.356794+0000","last_active":"2024-09-15T21:18:41.356794+0000","last_peered":"2024-09-15T21:18:36.376338+0000","last_clean":"2024-09-15T21:18:36.376338+0000","last_became_active":"2024-09-15T21:18:31.675622+0000","last_became_peered":"2024-09-15T21:18:31.675622+0000","last_unstale":"2024-09-15T21:18:36.376338+0000","last_undegraded":"2024-09-15T21:18:36.376338+0000","last_fullsized":"2024-09-15T21:18:36.376338+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":90,"reported_epoch":41,"state":"active+clean+remapped","last_fresh":"2024-09-15T21:18:36.377709+0000","last_change":"2024-09-15T21:18:31.716253+0000","last_active":"2024-09-15T21:18:36.377709+0000","last_peered":"2024-09-15T21:18:36.377709+0000","last_clean":"2024-09-15T21:18:36.377709+0000","last_became_active":"2024-09-15T21:18:31.713938+0000","last_became_peered":"2024-09-15T21:18:31.713938+0000","last_unstale":"2024-09-15T21:18:36.377709+0000","last_undegraded":"2024-09-15T21:18:36.377709+0000","last_fullsized":"2024-09-15T21:18:36.377709+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:20.500541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"41'3482","reported_seq":3597,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-15T21:18:41.321065+0000","last_change":"2024-09-15T21:18:41.355200+0000","last_active":"2024-09-15T21:18:41.355200+0000","last_peered":"2024-09-15T21:18:41.321065+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:31.676228+0000","last_became_peered":"2024-09-15T21:18:31.676228+0000","last_unstale":"2024-09-15T21:18:41.321065+0000","last_undegraded":"2024-09-15T21:18:41.321065+0000","last_fullsized":"2024-09-15T21:18:41.321065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3482,"log_dups_size":0,"ondisk_log_size":3482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14213737,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1142,"num_write_kb":4568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222315113,"num_objects":54277,"num_object_clones":0,"num_object_copies":108554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25772,"num_write_kb":103088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":444723200,"data_stored":444636225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54358,"ondisk_log_size":54358,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325512,"kb_used_data":217456,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94046328,"statfs":{"total":96636764160,"available":96303439872,"internally_reserved":0,"allocated":222674944,"data_stored":222424661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":110620333},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149717,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319308,"kb_used_data":215484,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94052532,"statfs":{"total":96636764160,"available":96309792768,"internally_reserved":0,"allocated":220655616,"data_stored":220414813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":106294961},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":462848,"data_stored":128112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":32435884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222318592,"data_stored":222315113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":14191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222310400,"data_stored":222306921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:47.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:48.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 55 20 56992 56972 4.04559 2.55078 0.0432087 0.0193008 2024-09-15T21:18:48.500 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:48.536 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-15T21:18:48.307101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":226546300,"num_objects":55312,"num_object_clones":0,"num_object_copies":110624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26807,"num_write_kb":107222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55394,"ondisk_log_size":55394,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":745296,"kb_used_data":443416,"kb_used_omap":12,"kb_used_meta":301811,"kb_avail":282370224,"statfs":{"total":289910292480,"available":289147109376,"internally_reserved":0,"allocated":454057984,"data_stored":453220263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":309054474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,7,13,5,3,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18980864,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":-37814,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4634,"num_write_kb":18536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.958116"},"pg_stats":[{"pgid":"2.f","version":"44'3452","reported_seq":3523,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.525060+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:46.525060+0000","last_peered":"2024-09-15T21:18:46.525060+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:46.525060+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3452,"log_dups_size":0,"ondisk_log_size":3452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3452,"num_write_kb":13808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3419","reported_seq":3510,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371973+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:43.371973+0000","last_peered":"2024-09-15T21:18:43.371973+0000","last_clean":"2024-09-15T21:18:43.371973+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:43.371973+0000","last_undegraded":"2024-09-15T21:18:43.371973+0000","last_fullsized":"2024-09-15T21:18:43.371973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3419,"log_dups_size":0,"ondisk_log_size":3419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3396","reported_seq":3507,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.372668+0000","last_change":"2024-09-15T21:18:43.436365+0000","last_active":"2024-09-15T21:18:41.362222+0000","last_peered":"2024-09-15T21:18:41.353008+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:31.680292+0000","last_became_peered":"2024-09-15T21:18:31.680292+0000","last_unstale":"2024-09-15T21:18:43.372668+0000","last_undegraded":"2024-09-15T21:18:43.372668+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3396,"log_dups_size":0,"ondisk_log_size":3396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893632,"num_objects":3392,"num_object_clones":0,"num_object_copies":6784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2119,"num_write_kb":8476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'3462","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371797+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:43.371797+0000","last_peered":"2024-09-15T21:18:43.371797+0000","last_clean":"2024-09-15T21:18:43.371797+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:43.371797+0000","last_undegraded":"2024-09-15T21:18:43.371797+0000","last_fullsized":"2024-09-15T21:18:43.371797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3462,"log_dups_size":0,"ondisk_log_size":3462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14180352,"num_objects":3462,"num_object_clones":0,"num_object_copies":6924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3536","reported_seq":3649,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:46.582647+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:46.582647+0000","last_peered":"2024-09-15T21:18:46.582647+0000","last_clean":"2024-09-15T21:18:46.582647+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:46.582647+0000","last_undegraded":"2024-09-15T21:18:46.582647+0000","last_fullsized":"2024-09-15T21:18:46.582647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14454784,"num_objects":3529,"num_object_clones":0,"num_object_copies":7058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3487","reported_seq":3609,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.608331+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:46.608331+0000","last_peered":"2024-09-15T21:18:46.608331+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:46.608331+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3487,"log_dups_size":0,"ondisk_log_size":3487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14262272,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":111},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3595","reported_seq":3713,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.567978+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:46.567978+0000","last_peered":"2024-09-15T21:18:46.567978+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:46.567978+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14700544,"num_objects":3589,"num_object_clones":0,"num_object_copies":7178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":118},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3551","reported_seq":3665,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:46.602353+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:46.602353+0000","last_peered":"2024-09-15T21:18:46.602353+0000","last_clean":"2024-09-15T21:18:46.602353+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:46.602353+0000","last_undegraded":"2024-09-15T21:18:46.602353+0000","last_fullsized":"2024-09-15T21:18:46.602353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1336,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'3345","reported_seq":3460,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369078+0000","last_change":"2024-09-15T21:18:43.442399+0000","last_active":"2024-09-15T21:18:41.362286+0000","last_peered":"2024-09-15T21:18:41.351933+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:31.682424+0000","last_became_peered":"2024-09-15T21:18:31.682424+0000","last_unstale":"2024-09-15T21:18:43.369078+0000","last_undegraded":"2024-09-15T21:18:43.369078+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3345,"log_dups_size":0,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3456","reported_seq":3573,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.607849+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:46.607849+0000","last_peered":"2024-09-15T21:18:46.607849+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:46.607849+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3456,"log_dups_size":0,"ondisk_log_size":3456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14123008,"num_objects":3448,"num_object_clones":0,"num_object_copies":6896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":982,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3271","reported_seq":3371,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369147+0000","last_change":"2024-09-15T21:18:43.442812+0000","last_active":"2024-09-15T21:18:41.354478+0000","last_peered":"2024-09-15T21:18:41.345310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:31.708127+0000","last_became_peered":"2024-09-15T21:18:31.708127+0000","last_unstale":"2024-09-15T21:18:43.369147+0000","last_undegraded":"2024-09-15T21:18:43.369147+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3271,"log_dups_size":0,"ondisk_log_size":3271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13377536,"num_objects":3266,"num_object_clones":0,"num_object_copies":6532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3266}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'3356","reported_seq":3461,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.368584+0000","last_change":"2024-09-15T21:18:43.448670+0000","last_active":"2024-09-15T21:18:41.357225+0000","last_peered":"2024-09-15T21:18:41.344494+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:31.696760+0000","last_became_peered":"2024-09-15T21:18:31.696760+0000","last_unstale":"2024-09-15T21:18:43.368584+0000","last_undegraded":"2024-09-15T21:18:43.368584+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3356,"log_dups_size":0,"ondisk_log_size":3356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13721600,"num_objects":3350,"num_object_clones":0,"num_object_copies":6700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":7020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3528","reported_seq":3650,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.636084+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:46.636084+0000","last_peered":"2024-09-15T21:18:46.636084+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:46.636084+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3528,"log_dups_size":0,"ondisk_log_size":3528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442496,"num_objects":3526,"num_object_clones":0,"num_object_copies":7052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":835,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3491","reported_seq":3615,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.550519+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:46.550519+0000","last_peered":"2024-09-15T21:18:46.550519+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:46.550519+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3491,"log_dups_size":0,"ondisk_log_size":3491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14266368,"num_objects":3483,"num_object_clones":0,"num_object_copies":6966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1076,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":104},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3453","reported_seq":3571,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.588582+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:46.588582+0000","last_peered":"2024-09-15T21:18:46.588582+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:46.588582+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3453,"log_dups_size":0,"ondisk_log_size":3453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14127104,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1100,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":84},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3593","reported_seq":3716,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.620490+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:46.620490+0000","last_peered":"2024-09-15T21:18:46.620490+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:46.620490+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3593,"log_dups_size":0,"ondisk_log_size":3593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14668393,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1253,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":112},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":226546281,"num_objects":55310,"num_object_clones":0,"num_object_copies":110620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26805,"num_write_kb":107220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454696960,"data_stored":454609985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55391,"ondisk_log_size":55391,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325512,"kb_used_data":217456,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94046328,"statfs":{"total":96636764160,"available":96303439872,"internally_reserved":0,"allocated":222674944,"data_stored":222424661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":110620333},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149718,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348392,"kb_used_data":222360,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94023448,"statfs":{"total":96636764160,"available":96280010752,"internally_reserved":0,"allocated":227696640,"data_stored":227441190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":129035953},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626435,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71392,"kb_used_data":3600,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300448,"statfs":{"total":96636764160,"available":96563658752,"internally_reserved":0,"allocated":3686400,"data_stored":3354412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":69398188},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":227360768,"data_stored":227357289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5025792,"data_stored":4945775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222310400,"data_stored":222306921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:48.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:18:48.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'3452", 'reported_seq': 3523, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.525060+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:18:46.525060+0000', 'last_peered': '2024-09-15T21:18:46.525060+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:18:46.525060+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 3452, 'log_dups_size': 0, 'ondisk_log_size': 3452, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14139392, 'num_objects': 3452, 'num_object_clones': 0, 'num_object_copies': 6904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3452, 'num_write_kb': 13808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 85}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:18:48.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "41'3396", 'reported_seq': 3507, 'reported_epoch': 44, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-15T21:18:43.372668+0000', 'last_change': '2024-09-15T21:18:43.436365+0000', 'last_active': '2024-09-15T21:18:41.362222+0000', 'last_peered': '2024-09-15T21:18:41.353008+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:31.680292+0000', 'last_became_peered': '2024-09-15T21:18:31.680292+0000', 'last_unstale': '2024-09-15T21:18:43.372668+0000', 'last_undegraded': '2024-09-15T21:18:43.372668+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 3396, 'log_dups_size': 0, 'ondisk_log_size': 3396, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13893632, 'num_objects': 3392, 'num_object_clones': 0, 'num_object_copies': 6784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3393, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2119, 'num_write_kb': 8476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'3487", 'reported_seq': 3609, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.608331+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:18:46.608331+0000', 'last_peered': '2024-09-15T21:18:46.608331+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:18:46.608331+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 3487, 'log_dups_size': 0, 'ondisk_log_size': 3487, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14262272, 'num_objects': 3482, 'num_object_clones': 0, 'num_object_copies': 6964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 912, 'num_write_kb': 3648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 111}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'3595", 'reported_seq': 3713, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.567978+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:18:46.567978+0000', 'last_peered': '2024-09-15T21:18:46.567978+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:18:46.567978+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 3595, 'log_dups_size': 0, 'ondisk_log_size': 3595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14700544, 'num_objects': 3589, 'num_object_clones': 0, 'num_object_copies': 7178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1231, 'num_write_kb': 4924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 118}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "41'3345", 'reported_seq': 3460, 'reported_epoch': 44, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-15T21:18:43.369078+0000', 'last_change': '2024-09-15T21:18:43.442399+0000', 'last_active': '2024-09-15T21:18:41.362286+0000', 'last_peered': '2024-09-15T21:18:41.351933+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:31.682424+0000', 'last_became_peered': '2024-09-15T21:18:31.682424+0000', 'last_unstale': '2024-09-15T21:18:43.369078+0000', 'last_undegraded': '2024-09-15T21:18:43.369078+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 3345, 'log_dups_size': 0, 'ondisk_log_size': 3345, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13672448, 'num_objects': 3338, 'num_object_clones': 0, 'num_object_copies': 6676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1931, 'num_write_kb': 7724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3338}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'3456", 'reported_seq': 3573, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.607849+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:18:46.607849+0000', 'last_peered': '2024-09-15T21:18:46.607849+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:18:46.607849+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 3456, 'log_dups_size': 0, 'ondisk_log_size': 3456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14123008, 'num_objects': 3448, 'num_object_clones': 0, 'num_object_copies': 6896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 982, 'num_write_kb': 3928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 107}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:18:48.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "41'3271", 'reported_seq': 3371, 'reported_epoch': 44, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-15T21:18:43.369147+0000', 'last_change': '2024-09-15T21:18:43.442812+0000', 'last_active': '2024-09-15T21:18:41.354478+0000', 'last_peered': '2024-09-15T21:18:41.345310+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:31.708127+0000', 'last_became_peered': '2024-09-15T21:18:31.708127+0000', 'last_unstale': '2024-09-15T21:18:43.369147+0000', 'last_undegraded': '2024-09-15T21:18:43.369147+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 3271, 'log_dups_size': 0, 'ondisk_log_size': 3271, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13377536, 'num_objects': 3266, 'num_object_clones': 0, 'num_object_copies': 6532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1822, 'num_write_kb': 7288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3266}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "41'3356", 'reported_seq': 3461, 'reported_epoch': 44, 'state': 'activating+undersized+remapped', 'last_fresh': '2024-09-15T21:18:43.368584+0000', 'last_change': '2024-09-15T21:18:43.448670+0000', 'last_active': '2024-09-15T21:18:41.357225+0000', 'last_peered': '2024-09-15T21:18:41.344494+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:31.696760+0000', 'last_became_peered': '2024-09-15T21:18:31.696760+0000', 'last_unstale': '2024-09-15T21:18:43.368584+0000', 'last_undegraded': '2024-09-15T21:18:43.368584+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 3356, 'log_dups_size': 0, 'ondisk_log_size': 3356, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13721600, 'num_objects': 3350, 'num_object_clones': 0, 'num_object_copies': 6700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1755, 'num_write_kb': 7020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3350}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'3528", 'reported_seq': 3650, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.636084+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:18:46.636084+0000', 'last_peered': '2024-09-15T21:18:46.636084+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:18:46.636084+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 3528, 'log_dups_size': 0, 'ondisk_log_size': 3528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14442496, 'num_objects': 3526, 'num_object_clones': 0, 'num_object_copies': 7052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 835, 'num_write_kb': 3340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 93}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'3491", 'reported_seq': 3615, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.550519+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:18:46.550519+0000', 'last_peered': '2024-09-15T21:18:46.550519+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:18:46.550519+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 3491, 'log_dups_size': 0, 'ondisk_log_size': 3491, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14266368, 'num_objects': 3483, 'num_object_clones': 0, 'num_object_copies': 6966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1076, 'num_write_kb': 4304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 104}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'3453", 'reported_seq': 3571, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.588582+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:18:46.588582+0000', 'last_peered': '2024-09-15T21:18:46.588582+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:18:46.588582+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 3453, 'log_dups_size': 0, 'ondisk_log_size': 3453, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14127104, 'num_objects': 3449, 'num_object_clones': 0, 'num_object_copies': 6898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1100, 'num_write_kb': 4400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 84}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:18:48.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'3593", 'reported_seq': 3716, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:46.620490+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:18:46.620490+0000', 'last_peered': '2024-09-15T21:18:46.620490+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:18:46.620490+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 3593, 'log_dups_size': 0, 'ondisk_log_size': 3593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14668393, 'num_objects': 3582, 'num_object_clones': 0, 'num_object_copies': 7164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1253, 'num_write_kb': 5012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 112}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:48.541 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:18:49.478 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 56 20 57641 57621 4.01862 2.53516 0.00699525 0.0194284 2024-09-15T21:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:49.575 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40565/110624 objects degraded (36.669%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"activating+undersized+remapped","count":4}],"num_pgs":24,"num_pools":2,"num_objects":55312,"data_bytes":226546300,"bytes_used":763183104,"bytes_avail":289147109376,"bytes_total":289910292480,"inactive_pgs_ratio":0.1666666716337204,"degraded_objects":40565,"degraded_total":110624,"degraded_ratio":0.36669258027191204,"write_bytes_sec":2727874,"read_op_per_sec":0,"write_op_per_sec":665},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:18:49.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-15T21:18:49.643 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:50.260 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:50.261 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:50.327 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-15T21:18:48.307101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":226546300,"num_objects":55312,"num_object_clones":0,"num_object_copies":110624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26807,"num_write_kb":107222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55394,"ondisk_log_size":55394,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":745296,"kb_used_data":443416,"kb_used_omap":12,"kb_used_meta":301811,"kb_avail":282370224,"statfs":{"total":289910292480,"available":289147109376,"internally_reserved":0,"allocated":454057984,"data_stored":453220263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":309054474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,7,13,5,3,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18980864,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":-37814,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4634,"num_write_kb":18536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.958116"},"pg_stats":[{"pgid":"2.f","version":"44'3452","reported_seq":3523,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.525060+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:46.525060+0000","last_peered":"2024-09-15T21:18:46.525060+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:46.525060+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3452,"log_dups_size":0,"ondisk_log_size":3452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14139392,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3452,"num_write_kb":13808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":85},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3419","reported_seq":3510,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371973+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:43.371973+0000","last_peered":"2024-09-15T21:18:43.371973+0000","last_clean":"2024-09-15T21:18:43.371973+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:43.371973+0000","last_undegraded":"2024-09-15T21:18:43.371973+0000","last_fullsized":"2024-09-15T21:18:43.371973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3419,"log_dups_size":0,"ondisk_log_size":3419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'3396","reported_seq":3507,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.372668+0000","last_change":"2024-09-15T21:18:43.436365+0000","last_active":"2024-09-15T21:18:41.362222+0000","last_peered":"2024-09-15T21:18:41.353008+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:31.680292+0000","last_became_peered":"2024-09-15T21:18:31.680292+0000","last_unstale":"2024-09-15T21:18:43.372668+0000","last_undegraded":"2024-09-15T21:18:43.372668+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3396,"log_dups_size":0,"ondisk_log_size":3396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893632,"num_objects":3392,"num_object_clones":0,"num_object_copies":6784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2119,"num_write_kb":8476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'3462","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.371797+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:43.371797+0000","last_peered":"2024-09-15T21:18:43.371797+0000","last_clean":"2024-09-15T21:18:43.371797+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:43.371797+0000","last_undegraded":"2024-09-15T21:18:43.371797+0000","last_fullsized":"2024-09-15T21:18:43.371797+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3462,"log_dups_size":0,"ondisk_log_size":3462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14180352,"num_objects":3462,"num_object_clones":0,"num_object_copies":6924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3536","reported_seq":3649,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:46.582647+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:46.582647+0000","last_peered":"2024-09-15T21:18:46.582647+0000","last_clean":"2024-09-15T21:18:46.582647+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:46.582647+0000","last_undegraded":"2024-09-15T21:18:46.582647+0000","last_fullsized":"2024-09-15T21:18:46.582647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14454784,"num_objects":3529,"num_object_clones":0,"num_object_copies":7058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3487","reported_seq":3609,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.608331+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:46.608331+0000","last_peered":"2024-09-15T21:18:46.608331+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:46.608331+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3487,"log_dups_size":0,"ondisk_log_size":3487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14262272,"num_objects":3482,"num_object_clones":0,"num_object_copies":6964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":111},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3595","reported_seq":3713,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.567978+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:46.567978+0000","last_peered":"2024-09-15T21:18:46.567978+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:46.567978+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14700544,"num_objects":3589,"num_object_clones":0,"num_object_copies":7178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":118},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3551","reported_seq":3665,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:46.602353+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:46.602353+0000","last_peered":"2024-09-15T21:18:46.602353+0000","last_clean":"2024-09-15T21:18:46.602353+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:46.602353+0000","last_undegraded":"2024-09-15T21:18:46.602353+0000","last_fullsized":"2024-09-15T21:18:46.602353+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3551,"log_dups_size":0,"ondisk_log_size":3551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14536704,"num_objects":3549,"num_object_clones":0,"num_object_copies":7098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1336,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'3345","reported_seq":3460,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369078+0000","last_change":"2024-09-15T21:18:43.442399+0000","last_active":"2024-09-15T21:18:41.362286+0000","last_peered":"2024-09-15T21:18:41.351933+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:31.682424+0000","last_became_peered":"2024-09-15T21:18:31.682424+0000","last_unstale":"2024-09-15T21:18:43.369078+0000","last_undegraded":"2024-09-15T21:18:43.369078+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3345,"log_dups_size":0,"ondisk_log_size":3345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13672448,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1931,"num_write_kb":7724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3456","reported_seq":3573,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.607849+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:46.607849+0000","last_peered":"2024-09-15T21:18:46.607849+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:46.607849+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3456,"log_dups_size":0,"ondisk_log_size":3456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14123008,"num_objects":3448,"num_object_clones":0,"num_object_copies":6896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":982,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'3271","reported_seq":3371,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.369147+0000","last_change":"2024-09-15T21:18:43.442812+0000","last_active":"2024-09-15T21:18:41.354478+0000","last_peered":"2024-09-15T21:18:41.345310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:31.708127+0000","last_became_peered":"2024-09-15T21:18:31.708127+0000","last_unstale":"2024-09-15T21:18:43.369147+0000","last_undegraded":"2024-09-15T21:18:43.369147+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3271,"log_dups_size":0,"ondisk_log_size":3271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13377536,"num_objects":3266,"num_object_clones":0,"num_object_copies":6532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3266}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'3356","reported_seq":3461,"reported_epoch":44,"state":"activating+undersized+remapped","last_fresh":"2024-09-15T21:18:43.368584+0000","last_change":"2024-09-15T21:18:43.448670+0000","last_active":"2024-09-15T21:18:41.357225+0000","last_peered":"2024-09-15T21:18:41.344494+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:31.696760+0000","last_became_peered":"2024-09-15T21:18:31.696760+0000","last_unstale":"2024-09-15T21:18:43.368584+0000","last_undegraded":"2024-09-15T21:18:43.368584+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3356,"log_dups_size":0,"ondisk_log_size":3356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13721600,"num_objects":3350,"num_object_clones":0,"num_object_copies":6700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1755,"num_write_kb":7020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3350}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3528","reported_seq":3650,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.636084+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:46.636084+0000","last_peered":"2024-09-15T21:18:46.636084+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:46.636084+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3528,"log_dups_size":0,"ondisk_log_size":3528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442496,"num_objects":3526,"num_object_clones":0,"num_object_copies":7052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":835,"num_write_kb":3340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3491","reported_seq":3615,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.550519+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:46.550519+0000","last_peered":"2024-09-15T21:18:46.550519+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:46.550519+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3491,"log_dups_size":0,"ondisk_log_size":3491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14266368,"num_objects":3483,"num_object_clones":0,"num_object_copies":6966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1076,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":104},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3453","reported_seq":3571,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.588582+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:46.588582+0000","last_peered":"2024-09-15T21:18:46.588582+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:46.588582+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3453,"log_dups_size":0,"ondisk_log_size":3453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14127104,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1100,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":84},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3593","reported_seq":3716,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:46.620490+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:46.620490+0000","last_peered":"2024-09-15T21:18:46.620490+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:46.620490+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3593,"log_dups_size":0,"ondisk_log_size":3593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14668393,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1253,"num_write_kb":5012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":112},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":226546281,"num_objects":55310,"num_object_clones":0,"num_object_copies":110620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26805,"num_write_kb":107220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454696960,"data_stored":454609985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55391,"ondisk_log_size":55391,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325512,"kb_used_data":217456,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94046328,"statfs":{"total":96636764160,"available":96303439872,"internally_reserved":0,"allocated":222674944,"data_stored":222424661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":110620333},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149718,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":348392,"kb_used_data":222360,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94023448,"statfs":{"total":96636764160,"available":96280010752,"internally_reserved":0,"allocated":227696640,"data_stored":227441190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":129035953},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626435,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71392,"kb_used_data":3600,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300448,"statfs":{"total":96636764160,"available":96563658752,"internally_reserved":0,"allocated":3686400,"data_stored":3354412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":69398188},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":227360768,"data_stored":227357289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5025792,"data_stored":4945775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222310400,"data_stored":222306921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:50.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 57 20 58276 58256 3.99163 2.48047 0.0569488 0.0195617 2024-09-15T21:18:51.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 58 20 58884 58864 3.96376 2.375 0.0486409 0.0197003 2024-09-15T21:18:52.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 59 20 59577 59557 3.94245 2.70703 0.00742097 0.0198082 2024-09-15T21:18:53.329 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:53.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:18:53.477940+0000 min lat: 0.00128075 max lat: 2.56993 avg lat: 0.0199038 2024-09-15T21:18:53.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:18:53.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 60 20 60287 60267 3.92297 2.77344 0.011562 0.0199038 2024-09-15T21:18:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:54.161 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-15T21:18:52.309906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238723708,"num_objects":58285,"num_object_clones":0,"num_object_copies":116570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":58285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29780,"num_write_kb":119114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":651283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58367,"ondisk_log_size":58367,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":805964,"kb_used_data":465244,"kb_used_omap":12,"kb_used_meta":340659,"kb_avail":282309556,"statfs":{"total":289910292480,"available":289084985344,"internally_reserved":0,"allocated":476409856,"data_stored":475566968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":348834826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,5,8,9,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19951616,"num_objects":4871,"num_object_clones":0,"num_object_copies":9742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":-9552,"num_objects_unfound":0,"num_objects_dirty":4871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4871,"num_write_kb":19484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.948728"},"pg_stats":[{"pgid":"2.f","version":"44'3641","reported_seq":3712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.616069+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:51.616069+0000","last_peered":"2024-09-15T21:18:51.616069+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:51.616069+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3641,"log_dups_size":0,"ondisk_log_size":3641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14913536,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3641,"num_write_kb":14564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":274},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3580","reported_seq":3671,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:48.700942+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:48.700942+0000","last_peered":"2024-09-15T21:18:48.700942+0000","last_clean":"2024-09-15T21:18:48.700942+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:48.700942+0000","last_undegraded":"2024-09-15T21:18:48.700942+0000","last_fullsized":"2024-09-15T21:18:48.700942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14639104,"num_objects":3574,"num_object_clones":0,"num_object_copies":7148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3557","reported_seq":3989,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:48.711708+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:48.711708+0000","last_peered":"2024-09-15T21:18:48.711708+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:48.711708+0000","last_undegraded":"2024-09-15T21:18:48.711708+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14553088,"num_objects":3553,"num_object_clones":0,"num_object_copies":7106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":3553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2280,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3233},{"shards":"2","objects":320}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'3644","reported_seq":3708,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:48.660657+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:48.660657+0000","last_peered":"2024-09-15T21:18:48.660657+0000","last_clean":"2024-09-15T21:18:48.660657+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:48.660657+0000","last_undegraded":"2024-09-15T21:18:48.660657+0000","last_fullsized":"2024-09-15T21:18:48.660657+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14925824,"num_objects":3644,"num_object_clones":0,"num_object_copies":7288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3725","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.626643+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:51.626643+0000","last_peered":"2024-09-15T21:18:51.626643+0000","last_clean":"2024-09-15T21:18:51.626643+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:51.626643+0000","last_undegraded":"2024-09-15T21:18:51.626643+0000","last_fullsized":"2024-09-15T21:18:51.626643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3725,"log_dups_size":0,"ondisk_log_size":3725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1583,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3676","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.627957+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:51.627957+0000","last_peered":"2024-09-15T21:18:51.627957+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:51.627957+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15036416,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1101,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":300},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3786","reported_seq":3904,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.611348+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:51.611348+0000","last_peered":"2024-09-15T21:18:51.611348+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:51.611348+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3786,"log_dups_size":0,"ondisk_log_size":3786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15482880,"num_objects":3780,"num_object_clones":0,"num_object_copies":7560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1422,"num_write_kb":5688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3724","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.605211+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:51.605211+0000","last_peered":"2024-09-15T21:18:51.605211+0000","last_clean":"2024-09-15T21:18:51.605211+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:51.605211+0000","last_undegraded":"2024-09-15T21:18:51.605211+0000","last_fullsized":"2024-09-15T21:18:51.605211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3724,"log_dups_size":0,"ondisk_log_size":3724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245312,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3522","reported_seq":3640,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:48.707974+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:48.707974+0000","last_peered":"2024-09-15T21:18:48.707974+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:48.707974+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14397440,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2108,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3671","reported_seq":3784,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.640966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:51.640966+0000","last_peered":"2024-09-15T21:18:51.640966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:51.640966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3671,"log_dups_size":0,"ondisk_log_size":3671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15003648,"num_objects":3663,"num_object_clones":0,"num_object_copies":7326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":322},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3478","reported_seq":3582,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:48.654510+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:48.654510+0000","last_peered":"2024-09-15T21:18:48.654510+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:48.654510+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3478,"log_dups_size":0,"ondisk_log_size":3478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14225408,"num_objects":3473,"num_object_clones":0,"num_object_copies":6946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2029,"num_write_kb":8116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3518","reported_seq":3627,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:48.689526+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:48.689526+0000","last_peered":"2024-09-15T21:18:48.689526+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:48.689526+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3518,"log_dups_size":0,"ondisk_log_size":3518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14385152,"num_objects":3512,"num_object_clones":0,"num_object_copies":7024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1917,"num_write_kb":7668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":162}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3728","reported_seq":3851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.610765+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:51.610765+0000","last_peered":"2024-09-15T21:18:51.610765+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:51.610765+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3728,"log_dups_size":0,"ondisk_log_size":3728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":293},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3697","reported_seq":3821,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.643759+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:51.643759+0000","last_peered":"2024-09-15T21:18:51.643759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:51.643759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1282,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":310},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3644","reported_seq":3761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.645364+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:51.645364+0000","last_peered":"2024-09-15T21:18:51.645364+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:51.645364+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":5164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3773","reported_seq":3896,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.623237+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:51.623237+0000","last_peered":"2024-09-15T21:18:51.623237+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:51.623237+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3773,"log_dups_size":0,"ondisk_log_size":3773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405673,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1433,"num_write_kb":5732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":292},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":238723689,"num_objects":58283,"num_object_clones":0,"num_object_copies":116566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3233,"num_objects_unfound":0,"num_objects_dirty":58283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29778,"num_write_kb":119112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":651264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":480251904,"data_stored":480164929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58364,"ondisk_log_size":58364,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953491,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352332,"kb_used_data":222076,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94019508,"statfs":{"total":96636764160,"available":96275976192,"internally_reserved":0,"allocated":227405824,"data_stored":227150374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":133361325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364228,"kb_used_data":229876,"kb_used_omap":4,"kb_used_meta":134331,"kb_avail":94007612,"statfs":{"total":96636764160,"available":96263794688,"internally_reserved":0,"allocated":235393024,"data_stored":235137574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":137555633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,5,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89404,"kb_used_data":13292,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94282436,"statfs":{"total":96636764160,"available":96545214464,"internally_reserved":0,"allocated":13611008,"data_stored":13279020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236843008,"data_stored":236839529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15228928,"data_stored":15148911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228179968,"data_stored":228176489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:54.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:54.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 61 20 60849 60829 3.89464 2.19531 0.0305241 0.0200488 2024-09-15T21:18:54.818 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:54.821 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:54.865 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-15T21:18:54.310258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243909244,"num_objects":59551,"num_object_clones":0,"num_object_copies":119102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":59551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31046,"num_write_kb":124178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1179667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59633,"ondisk_log_size":59633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":800080,"kb_used_data":471840,"kb_used_omap":12,"kb_used_meta":328179,"kb_avail":282315440,"statfs":{"total":289910292480,"available":289091010560,"internally_reserved":0,"allocated":483164160,"data_stored":482321272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":336055306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,5,9,10,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":-9681,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.946970"},"pg_stats":[{"pgid":"2.f","version":"44'3641","reported_seq":3712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.616069+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:51.616069+0000","last_peered":"2024-09-15T21:18:51.616069+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:51.616069+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3641,"log_dups_size":0,"ondisk_log_size":3641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14913536,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3641,"num_write_kb":14564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":274},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3793","reported_seq":3884,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.699227+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:53.699227+0000","last_peered":"2024-09-15T21:18:53.699227+0000","last_clean":"2024-09-15T21:18:53.699227+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:53.699227+0000","last_undegraded":"2024-09-15T21:18:53.699227+0000","last_fullsized":"2024-09-15T21:18:53.699227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15511552,"num_objects":3787,"num_object_clones":0,"num_object_copies":7574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3746","reported_seq":4438,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:53.688096+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:53.688096+0000","last_peered":"2024-09-15T21:18:53.688096+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:53.688096+0000","last_undegraded":"2024-09-15T21:18:53.688096+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3746,"log_dups_size":0,"ondisk_log_size":3746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3104},{"shards":"2","objects":638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'3875","reported_seq":3939,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.711900+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:53.711900+0000","last_peered":"2024-09-15T21:18:53.711900+0000","last_clean":"2024-09-15T21:18:53.711900+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:53.711900+0000","last_undegraded":"2024-09-15T21:18:53.711900+0000","last_fullsized":"2024-09-15T21:18:53.711900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3875,"log_dups_size":0,"ondisk_log_size":3875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872000,"num_objects":3875,"num_object_clones":0,"num_object_copies":7750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3875,"num_write_kb":15500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3725","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.626643+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:51.626643+0000","last_peered":"2024-09-15T21:18:51.626643+0000","last_clean":"2024-09-15T21:18:51.626643+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:51.626643+0000","last_undegraded":"2024-09-15T21:18:51.626643+0000","last_fullsized":"2024-09-15T21:18:51.626643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3725,"log_dups_size":0,"ondisk_log_size":3725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1583,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3676","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.627957+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:51.627957+0000","last_peered":"2024-09-15T21:18:51.627957+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:51.627957+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15036416,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1101,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":300},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3786","reported_seq":3904,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.611348+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:51.611348+0000","last_peered":"2024-09-15T21:18:51.611348+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:51.611348+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3786,"log_dups_size":0,"ondisk_log_size":3786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15482880,"num_objects":3780,"num_object_clones":0,"num_object_copies":7560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1422,"num_write_kb":5688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3724","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.605211+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:51.605211+0000","last_peered":"2024-09-15T21:18:51.605211+0000","last_clean":"2024-09-15T21:18:51.605211+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:51.605211+0000","last_undegraded":"2024-09-15T21:18:51.605211+0000","last_fullsized":"2024-09-15T21:18:51.605211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3724,"log_dups_size":0,"ondisk_log_size":3724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245312,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3730","reported_seq":3849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.673310+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:53.673310+0000","last_peered":"2024-09-15T21:18:53.673310+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:53.673310+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3671","reported_seq":3784,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.640966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:51.640966+0000","last_peered":"2024-09-15T21:18:51.640966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:51.640966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3671,"log_dups_size":0,"ondisk_log_size":3671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15003648,"num_objects":3663,"num_object_clones":0,"num_object_copies":7326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":322},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3694","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.678553+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:53.678553+0000","last_peered":"2024-09-15T21:18:53.678553+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:53.678553+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3727","reported_seq":3836,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.706310+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:53.706310+0000","last_peered":"2024-09-15T21:18:53.706310+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:53.706310+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3727,"log_dups_size":0,"ondisk_log_size":3727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241216,"num_objects":3721,"num_object_clones":0,"num_object_copies":7442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":8504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3728","reported_seq":3851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.610765+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:51.610765+0000","last_peered":"2024-09-15T21:18:51.610765+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:51.610765+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3728,"log_dups_size":0,"ondisk_log_size":3728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":293},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3697","reported_seq":3821,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.643759+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:51.643759+0000","last_peered":"2024-09-15T21:18:51.643759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:51.643759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1282,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":310},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3644","reported_seq":3761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.645364+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:51.645364+0000","last_peered":"2024-09-15T21:18:51.645364+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:51.645364+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":5164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3773","reported_seq":3896,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.623237+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:51.623237+0000","last_peered":"2024-09-15T21:18:51.623237+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:51.623237+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3773,"log_dups_size":0,"ondisk_log_size":3773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405673,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1433,"num_write_kb":5732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":292},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243909225,"num_objects":59549,"num_object_clones":0,"num_object_copies":119098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":59549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31044,"num_write_kb":124176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487002112,"data_stored":486915137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59630,"ondisk_log_size":59630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953492,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346448,"kb_used_data":228672,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94025392,"statfs":{"total":96636764160,"available":96282001408,"internally_reserved":0,"allocated":234160128,"data_stored":233904678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":120581805},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364228,"kb_used_data":229876,"kb_used_omap":4,"kb_used_meta":134331,"kb_avail":94007612,"statfs":{"total":96636764160,"available":96263794688,"internally_reserved":0,"allocated":235393024,"data_stored":235137574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":137555633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,5,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89404,"kb_used_data":13292,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94282436,"statfs":{"total":96636764160,"available":96545214464,"internally_reserved":0,"allocated":13611008,"data_stored":13279020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236843008,"data_stored":236839529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15228928,"data_stored":15148911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234930176,"data_stored":234926697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:54.866 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:18:54.866 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'3641", 'reported_seq': 3712, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.616069+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:18:51.616069+0000', 'last_peered': '2024-09-15T21:18:51.616069+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:18:51.616069+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 3641, 'log_dups_size': 0, 'ondisk_log_size': 3641, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14913536, 'num_objects': 3641, 'num_object_clones': 0, 'num_object_copies': 7282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3641, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3641, 'num_write_kb': 14564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 274}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.866 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:18:54.866 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'3746", 'reported_seq': 4438, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:18:53.688096+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:18:53.688096+0000', 'last_peered': '2024-09-15T21:18:53.688096+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:18:53.688096+0000', 'last_undegraded': '2024-09-15T21:18:53.688096+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 3746, 'log_dups_size': 0, 'ondisk_log_size': 3746, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15327232, 'num_objects': 3742, 'num_object_clones': 0, 'num_object_copies': 7484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3104, 'num_objects_unfound': 0, 'num_objects_dirty': 3742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2469, 'num_write_kb': 9876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 288, 'num_bytes_recovered': 1179648, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3104}, {'shards': '2', 'objects': 638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'3676", 'reported_seq': 3798, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.627957+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:18:51.627957+0000', 'last_peered': '2024-09-15T21:18:51.627957+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:18:51.627957+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 3676, 'log_dups_size': 0, 'ondisk_log_size': 3676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15036416, 'num_objects': 3671, 'num_object_clones': 0, 'num_object_copies': 7342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1101, 'num_write_kb': 4404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 300}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'3786", 'reported_seq': 3904, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.611348+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:18:51.611348+0000', 'last_peered': '2024-09-15T21:18:51.611348+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:18:51.611348+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 3786, 'log_dups_size': 0, 'ondisk_log_size': 3786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15482880, 'num_objects': 3780, 'num_object_clones': 0, 'num_object_copies': 7560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1422, 'num_write_kb': 5688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 309}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'3730", 'reported_seq': 3849, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:53.673310+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:18:53.673310+0000', 'last_peered': '2024-09-15T21:18:53.673310+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:18:53.673310+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 3730, 'log_dups_size': 0, 'ondisk_log_size': 3730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15249408, 'num_objects': 3723, 'num_object_clones': 0, 'num_object_copies': 7446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2316, 'num_write_kb': 9264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 385}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:18:54.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'3671", 'reported_seq': 3784, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.640966+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:18:51.640966+0000', 'last_peered': '2024-09-15T21:18:51.640966+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:18:51.640966+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 3671, 'log_dups_size': 0, 'ondisk_log_size': 3671, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15003648, 'num_objects': 3663, 'num_object_clones': 0, 'num_object_copies': 7326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1197, 'num_write_kb': 4788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 322}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'3694", 'reported_seq': 3798, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:53.678553+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:18:53.678553+0000', 'last_peered': '2024-09-15T21:18:53.678553+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:18:53.678553+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 3694, 'log_dups_size': 0, 'ondisk_log_size': 3694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15110144, 'num_objects': 3689, 'num_object_clones': 0, 'num_object_copies': 7378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2245, 'num_write_kb': 8980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 423}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'3727", 'reported_seq': 3836, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:53.706310+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:18:53.706310+0000', 'last_peered': '2024-09-15T21:18:53.706310+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:18:53.706310+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 3727, 'log_dups_size': 0, 'ondisk_log_size': 3727, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15241216, 'num_objects': 3721, 'num_object_clones': 0, 'num_object_copies': 7442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2126, 'num_write_kb': 8504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 371}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'3728", 'reported_seq': 3851, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.610765+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:18:51.610765+0000', 'last_peered': '2024-09-15T21:18:51.610765+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:18:51.610765+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 3728, 'log_dups_size': 0, 'ondisk_log_size': 3728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15261696, 'num_objects': 3726, 'num_object_clones': 0, 'num_object_copies': 7452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1035, 'num_write_kb': 4140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 293}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'3697", 'reported_seq': 3821, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.643759+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:18:51.643759+0000', 'last_peered': '2024-09-15T21:18:51.643759+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:18:51.643759+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 3697, 'log_dups_size': 0, 'ondisk_log_size': 3697, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15110144, 'num_objects': 3689, 'num_object_clones': 0, 'num_object_copies': 7378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1282, 'num_write_kb': 5128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 310}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:18:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'3644", 'reported_seq': 3761, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.645364+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:18:51.645364+0000', 'last_peered': '2024-09-15T21:18:51.645364+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:18:51.645364+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 3644, 'log_dups_size': 0, 'ondisk_log_size': 3644, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14909440, 'num_objects': 3640, 'num_object_clones': 0, 'num_object_copies': 7280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1291, 'num_write_kb': 5164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 275}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:18:54.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'3773", 'reported_seq': 3896, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:51.623237+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:18:51.623237+0000', 'last_peered': '2024-09-15T21:18:51.623237+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:18:51.623237+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 3773, 'log_dups_size': 0, 'ondisk_log_size': 3773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15405673, 'num_objects': 3762, 'num_object_clones': 0, 'num_object_copies': 7524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1433, 'num_write_kb': 5732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 292}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:18:54.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:18:55.416 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/119102 objects degraded (31.210%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":148,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59551,"data_bytes":243909244,"bytes_used":819281920,"bytes_avail":289091010560,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":119102,"degraded_ratio":0.31210223170055917,"misplaced_objects":3104,"misplaced_total":119102,"misplaced_ratio":0.026061695017715908,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":107760,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"write_bytes_sec":2296265,"read_op_per_sec":0,"write_op_per_sec":560},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:18:55.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:18:55.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:55.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 62 20 61298 61278 3.86012 1.75391 0.00809655 0.0201883 2024-09-15T21:18:56.124 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:56.125 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:56.223 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-15T21:18:54.310258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243909244,"num_objects":59551,"num_object_clones":0,"num_object_copies":119102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":59551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31046,"num_write_kb":124178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1179667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59633,"ondisk_log_size":59633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":800080,"kb_used_data":471840,"kb_used_omap":12,"kb_used_meta":328179,"kb_avail":282315440,"statfs":{"total":289910292480,"available":289091010560,"internally_reserved":0,"allocated":483164160,"data_stored":482321272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":336055306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,5,9,10,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":-9681,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.946970"},"pg_stats":[{"pgid":"2.f","version":"44'3641","reported_seq":3712,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.616069+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:51.616069+0000","last_peered":"2024-09-15T21:18:51.616069+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:51.616069+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3641,"log_dups_size":0,"ondisk_log_size":3641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14913536,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3641,"num_write_kb":14564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":274},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3793","reported_seq":3884,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.699227+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:53.699227+0000","last_peered":"2024-09-15T21:18:53.699227+0000","last_clean":"2024-09-15T21:18:53.699227+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:53.699227+0000","last_undegraded":"2024-09-15T21:18:53.699227+0000","last_fullsized":"2024-09-15T21:18:53.699227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15511552,"num_objects":3787,"num_object_clones":0,"num_object_copies":7574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3746","reported_seq":4438,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:53.688096+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:53.688096+0000","last_peered":"2024-09-15T21:18:53.688096+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:53.688096+0000","last_undegraded":"2024-09-15T21:18:53.688096+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3746,"log_dups_size":0,"ondisk_log_size":3746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3104},{"shards":"2","objects":638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'3875","reported_seq":3939,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.711900+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:53.711900+0000","last_peered":"2024-09-15T21:18:53.711900+0000","last_clean":"2024-09-15T21:18:53.711900+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:53.711900+0000","last_undegraded":"2024-09-15T21:18:53.711900+0000","last_fullsized":"2024-09-15T21:18:53.711900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3875,"log_dups_size":0,"ondisk_log_size":3875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872000,"num_objects":3875,"num_object_clones":0,"num_object_copies":7750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3875,"num_write_kb":15500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3725","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.626643+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:51.626643+0000","last_peered":"2024-09-15T21:18:51.626643+0000","last_clean":"2024-09-15T21:18:51.626643+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:51.626643+0000","last_undegraded":"2024-09-15T21:18:51.626643+0000","last_fullsized":"2024-09-15T21:18:51.626643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3725,"log_dups_size":0,"ondisk_log_size":3725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15228928,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1583,"num_write_kb":6332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3676","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.627957+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:51.627957+0000","last_peered":"2024-09-15T21:18:51.627957+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:51.627957+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15036416,"num_objects":3671,"num_object_clones":0,"num_object_copies":7342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1101,"num_write_kb":4404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":300},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3786","reported_seq":3904,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.611348+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:51.611348+0000","last_peered":"2024-09-15T21:18:51.611348+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:51.611348+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3786,"log_dups_size":0,"ondisk_log_size":3786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15482880,"num_objects":3780,"num_object_clones":0,"num_object_copies":7560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1422,"num_write_kb":5688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3724","reported_seq":3838,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:51.605211+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:51.605211+0000","last_peered":"2024-09-15T21:18:51.605211+0000","last_clean":"2024-09-15T21:18:51.605211+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:51.605211+0000","last_undegraded":"2024-09-15T21:18:51.605211+0000","last_fullsized":"2024-09-15T21:18:51.605211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3724,"log_dups_size":0,"ondisk_log_size":3724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245312,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1509,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3730","reported_seq":3849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.673310+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:53.673310+0000","last_peered":"2024-09-15T21:18:53.673310+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:53.673310+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3671","reported_seq":3784,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.640966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:51.640966+0000","last_peered":"2024-09-15T21:18:51.640966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:51.640966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3671,"log_dups_size":0,"ondisk_log_size":3671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15003648,"num_objects":3663,"num_object_clones":0,"num_object_copies":7326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":322},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3694","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.678553+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:53.678553+0000","last_peered":"2024-09-15T21:18:53.678553+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:53.678553+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3727","reported_seq":3836,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.706310+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:53.706310+0000","last_peered":"2024-09-15T21:18:53.706310+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:53.706310+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3727,"log_dups_size":0,"ondisk_log_size":3727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241216,"num_objects":3721,"num_object_clones":0,"num_object_copies":7442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":8504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3728","reported_seq":3851,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.610765+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:51.610765+0000","last_peered":"2024-09-15T21:18:51.610765+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:51.610765+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3728,"log_dups_size":0,"ondisk_log_size":3728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":293},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3697","reported_seq":3821,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.643759+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:51.643759+0000","last_peered":"2024-09-15T21:18:51.643759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:51.643759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1282,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":310},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3644","reported_seq":3761,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.645364+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:51.645364+0000","last_peered":"2024-09-15T21:18:51.645364+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:51.645364+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14909440,"num_objects":3640,"num_object_clones":0,"num_object_copies":7280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":5164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3773","reported_seq":3896,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:51.623237+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:51.623237+0000","last_peered":"2024-09-15T21:18:51.623237+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:51.623237+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3773,"log_dups_size":0,"ondisk_log_size":3773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405673,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1433,"num_write_kb":5732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":292},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243909225,"num_objects":59549,"num_object_clones":0,"num_object_copies":119098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":59549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31044,"num_write_kb":124176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487002112,"data_stored":486915137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59630,"ondisk_log_size":59630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953492,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346448,"kb_used_data":228672,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94025392,"statfs":{"total":96636764160,"available":96282001408,"internally_reserved":0,"allocated":234160128,"data_stored":233904678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":120581805},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149719,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364228,"kb_used_data":229876,"kb_used_omap":4,"kb_used_meta":134331,"kb_avail":94007612,"statfs":{"total":96636764160,"available":96263794688,"internally_reserved":0,"allocated":235393024,"data_stored":235137574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":137555633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,5,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626436,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89404,"kb_used_data":13292,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94282436,"statfs":{"total":96636764160,"available":96545214464,"internally_reserved":0,"allocated":13611008,"data_stored":13279020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236843008,"data_stored":236839529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15228928,"data_stored":15148911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234930176,"data_stored":234926697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:56.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 63 20 61919 61899 3.83735 2.42578 0.0528706 0.0203486 2024-09-15T21:18:57.479 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 64 20 62598 62578 3.81883 2.65234 0.0535342 0.0204489 2024-09-15T21:18:58.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 65 20 63287 63267 3.80139 2.69141 0.0822491 0.020539 2024-09-15T21:18:59.226 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:18:59.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 66 20 63884 63864 3.77913 2.33203 0.0262251 0.0206628 2024-09-15T21:18:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:18:59.859 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:18:59.922 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-15T21:18:58.316512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":251798140,"num_objects":61477,"num_object_clones":0,"num_object_copies":122954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":61477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32972,"num_write_kb":131882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1179667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61559,"ondisk_log_size":61559,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":805964,"kb_used_data":494364,"kb_used_omap":12,"kb_used_meta":311539,"kb_avail":282309556,"statfs":{"total":289910292480,"available":289084985344,"internally_reserved":0,"allocated":506228736,"data_stored":505385848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":319015946},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,7,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29483008,"num_objects":7198,"num_object_clones":0,"num_object_copies":14396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23819,"num_objects_misplaced":3103,"num_objects_unfound":0,"num_objects_dirty":7198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7198,"num_write_kb":28792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009740"},"pg_stats":[{"pgid":"2.f","version":"44'3829","reported_seq":3900,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.594944+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:56.594944+0000","last_peered":"2024-09-15T21:18:56.594944+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:56.594944+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3829,"num_write_kb":15316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3793","reported_seq":3884,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.699227+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:53.699227+0000","last_peered":"2024-09-15T21:18:53.699227+0000","last_clean":"2024-09-15T21:18:53.699227+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:53.699227+0000","last_undegraded":"2024-09-15T21:18:53.699227+0000","last_fullsized":"2024-09-15T21:18:53.699227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3793,"log_dups_size":0,"ondisk_log_size":3793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15511552,"num_objects":3787,"num_object_clones":0,"num_object_copies":7574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2519,"num_write_kb":10076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3746","reported_seq":4438,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:53.688096+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:53.688096+0000","last_peered":"2024-09-15T21:18:53.688096+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:53.688096+0000","last_undegraded":"2024-09-15T21:18:53.688096+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3746,"log_dups_size":0,"ondisk_log_size":3746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3104},{"shards":"2","objects":638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'3875","reported_seq":3939,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:53.711900+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:53.711900+0000","last_peered":"2024-09-15T21:18:53.711900+0000","last_clean":"2024-09-15T21:18:53.711900+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:53.711900+0000","last_undegraded":"2024-09-15T21:18:53.711900+0000","last_fullsized":"2024-09-15T21:18:53.711900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3875,"log_dups_size":0,"ondisk_log_size":3875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872000,"num_objects":3875,"num_object_clones":0,"num_object_copies":7750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3875,"num_write_kb":15500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3908","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.609290+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:56.609290+0000","last_peered":"2024-09-15T21:18:56.609290+0000","last_clean":"2024-09-15T21:18:56.609290+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:56.609290+0000","last_undegraded":"2024-09-15T21:18:56.609290+0000","last_fullsized":"2024-09-15T21:18:56.609290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3876","reported_seq":3997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.638568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:56.638568+0000","last_peered":"2024-09-15T21:18:56.638568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:56.638568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15855616,"num_objects":3871,"num_object_clones":0,"num_object_copies":7742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1301,"num_write_kb":5204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3943","reported_seq":4061,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.615694+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:56.615694+0000","last_peered":"2024-09-15T21:18:56.615694+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:56.615694+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3943,"log_dups_size":0,"ondisk_log_size":3943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16125952,"num_objects":3937,"num_object_clones":0,"num_object_copies":7874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3907","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.615285+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:56.615285+0000","last_peered":"2024-09-15T21:18:56.615285+0000","last_clean":"2024-09-15T21:18:56.615285+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:56.615285+0000","last_undegraded":"2024-09-15T21:18:56.615285+0000","last_fullsized":"2024-09-15T21:18:56.615285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3907,"log_dups_size":0,"ondisk_log_size":3907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15994880,"num_objects":3905,"num_object_clones":0,"num_object_copies":7810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1692,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3730","reported_seq":3849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.673310+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:53.673310+0000","last_peered":"2024-09-15T21:18:53.673310+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:53.673310+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3855","reported_seq":3972,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.640615+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:56.640615+0000","last_peered":"2024-09-15T21:18:56.640615+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:56.640615+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15757312,"num_objects":3847,"num_object_clones":0,"num_object_copies":7694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1381,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":506},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3694","reported_seq":3798,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.678553+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:53.678553+0000","last_peered":"2024-09-15T21:18:53.678553+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:53.678553+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3694,"log_dups_size":0,"ondisk_log_size":3694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15110144,"num_objects":3689,"num_object_clones":0,"num_object_copies":7378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":423}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3727","reported_seq":3836,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:53.706310+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:53.706310+0000","last_peered":"2024-09-15T21:18:53.706310+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:53.706310+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3727,"log_dups_size":0,"ondisk_log_size":3727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241216,"num_objects":3721,"num_object_clones":0,"num_object_copies":7442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2126,"num_write_kb":8504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3924","reported_seq":4047,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.647273+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:56.647273+0000","last_peered":"2024-09-15T21:18:56.647273+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:56.647273+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3924,"log_dups_size":0,"ondisk_log_size":3924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3908","reported_seq":4032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.677751+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:56.677751+0000","last_peered":"2024-09-15T21:18:56.677751+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:56.677751+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":521},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3856","reported_seq":3970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.661702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:56.661702+0000","last_peered":"2024-09-15T21:18:56.661702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:56.661702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3856,"log_dups_size":0,"ondisk_log_size":3856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1503,"num_write_kb":6012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":487},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3985","reported_seq":4108,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.665734+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:56.665734+0000","last_peered":"2024-09-15T21:18:56.665734+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:56.665734+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16274025,"num_objects":3974,"num_object_clones":0,"num_object_copies":7948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1645,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":504},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":251798121,"num_objects":61475,"num_object_clones":0,"num_object_copies":122950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":3104,"num_objects_unfound":0,"num_objects_dirty":61475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32970,"num_write_kb":131880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1179648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":506486784,"data_stored":506399809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61556,"ondisk_log_size":61556,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953492,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346448,"kb_used_data":228672,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94025392,"statfs":{"total":96636764160,"available":96282001408,"internally_reserved":0,"allocated":234160128,"data_stored":233904678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":120581805},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366876,"kb_used_data":240844,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94004964,"statfs":{"total":96636764160,"available":96261083136,"internally_reserved":0,"allocated":246624256,"data_stored":246368806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":129035953},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92640,"kb_used_data":24848,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94279200,"statfs":{"total":96636764160,"available":96541900800,"internally_reserved":0,"allocated":25444352,"data_stored":25112364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":69398188},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246419456,"data_stored":246415977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25137152,"data_stored":25057135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234930176,"data_stored":234926697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:18:59.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:00.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 67 20 64511 64491 3.75928 2.44922 0.00850234 0.0207694 2024-09-15T21:19:00.579 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:00.579 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:00.596 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-15T21:19:00.316971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":256209532,"num_objects":62554,"num_object_clones":0,"num_object_copies":125108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":62554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34049,"num_write_kb":136190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":1822739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62636,"ondisk_log_size":62636,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":816024,"kb_used_data":500264,"kb_used_omap":12,"kb_used_meta":315699,"kb_avail":282299496,"statfs":{"total":289910292480,"available":289074683904,"internally_reserved":0,"allocated":512270336,"data_stored":511427448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":323275786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,5,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29663232,"num_objects":7242,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3393,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7242,"num_write_kb":28968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009869"},"pg_stats":[{"pgid":"2.f","version":"44'3829","reported_seq":3900,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.594944+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:56.594944+0000","last_peered":"2024-09-15T21:18:56.594944+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:56.594944+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3829,"num_write_kb":15316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3968","reported_seq":4059,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:58.676547+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:58.676547+0000","last_peered":"2024-09-15T21:18:58.676547+0000","last_clean":"2024-09-15T21:18:58.676547+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:58.676547+0000","last_undegraded":"2024-09-15T21:18:58.676547+0000","last_fullsized":"2024-09-15T21:18:58.676547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3968,"log_dups_size":0,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2694,"num_write_kb":10776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3905","reported_seq":4911,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:58.676341+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:58.676341+0000","last_peered":"2024-09-15T21:18:58.676341+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:58.676341+0000","last_undegraded":"2024-09-15T21:18:58.676341+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3905,"log_dups_size":0,"ondisk_log_size":3905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2628,"num_write_kb":10512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2947},{"shards":"2","objects":954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4047","reported_seq":4111,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:58.644785+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:58.644785+0000","last_peered":"2024-09-15T21:18:58.644785+0000","last_clean":"2024-09-15T21:18:58.644785+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:58.644785+0000","last_undegraded":"2024-09-15T21:18:58.644785+0000","last_fullsized":"2024-09-15T21:18:58.644785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16576512,"num_objects":4047,"num_object_clones":0,"num_object_copies":8094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4047,"num_write_kb":16188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3908","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.609290+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:56.609290+0000","last_peered":"2024-09-15T21:18:56.609290+0000","last_clean":"2024-09-15T21:18:56.609290+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:56.609290+0000","last_undegraded":"2024-09-15T21:18:56.609290+0000","last_fullsized":"2024-09-15T21:18:56.609290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3876","reported_seq":3997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.638568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:56.638568+0000","last_peered":"2024-09-15T21:18:56.638568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:56.638568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15855616,"num_objects":3871,"num_object_clones":0,"num_object_copies":7742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1301,"num_write_kb":5204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3943","reported_seq":4061,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.615694+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:56.615694+0000","last_peered":"2024-09-15T21:18:56.615694+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:56.615694+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3943,"log_dups_size":0,"ondisk_log_size":3943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16125952,"num_objects":3937,"num_object_clones":0,"num_object_copies":7874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3907","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.615285+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:56.615285+0000","last_peered":"2024-09-15T21:18:56.615285+0000","last_clean":"2024-09-15T21:18:56.615285+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:56.615285+0000","last_undegraded":"2024-09-15T21:18:56.615285+0000","last_fullsized":"2024-09-15T21:18:56.615285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3907,"log_dups_size":0,"ondisk_log_size":3907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15994880,"num_objects":3905,"num_object_clones":0,"num_object_copies":7810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1692,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3913","reported_seq":4032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.679809+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:58.679809+0000","last_peered":"2024-09-15T21:18:58.679809+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:58.679809+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3855","reported_seq":3972,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.640615+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:56.640615+0000","last_peered":"2024-09-15T21:18:56.640615+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:56.640615+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15757312,"num_objects":3847,"num_object_clones":0,"num_object_copies":7694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1381,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":506},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3882","reported_seq":3986,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.699070+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:58.699070+0000","last_peered":"2024-09-15T21:18:58.699070+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:58.699070+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3882,"log_dups_size":0,"ondisk_log_size":3882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15880192,"num_objects":3877,"num_object_clones":0,"num_object_copies":7754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3927","reported_seq":4036,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.700346+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:58.700346+0000","last_peered":"2024-09-15T21:18:58.700346+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:58.700346+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3927,"log_dups_size":0,"ondisk_log_size":3927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2326,"num_write_kb":9304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3924","reported_seq":4047,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.647273+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:56.647273+0000","last_peered":"2024-09-15T21:18:56.647273+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:56.647273+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3924,"log_dups_size":0,"ondisk_log_size":3924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3908","reported_seq":4032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.677751+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:56.677751+0000","last_peered":"2024-09-15T21:18:56.677751+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:56.677751+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":521},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3856","reported_seq":3970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.661702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:56.661702+0000","last_peered":"2024-09-15T21:18:56.661702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:56.661702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3856,"log_dups_size":0,"ondisk_log_size":3856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1503,"num_write_kb":6012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":487},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3985","reported_seq":4108,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.665734+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:56.665734+0000","last_peered":"2024-09-15T21:18:56.665734+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:56.665734+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16274025,"num_objects":3974,"num_object_clones":0,"num_object_copies":7948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1645,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":504},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":256209513,"num_objects":62552,"num_object_clones":0,"num_object_copies":125104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":62552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34047,"num_write_kb":136188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512450560,"data_stored":512363585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62633,"ondisk_log_size":62633,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356508,"kb_used_data":234572,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94015332,"statfs":{"total":96636764160,"available":96271699968,"internally_reserved":0,"allocated":240201728,"data_stored":239946278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":124841645},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366876,"kb_used_data":240844,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94004964,"statfs":{"total":96636764160,"available":96261083136,"internally_reserved":0,"allocated":246624256,"data_stored":246368806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":129035953},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92640,"kb_used_data":24848,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94279200,"statfs":{"total":96636764160,"available":96541900800,"internally_reserved":0,"allocated":25444352,"data_stored":25112364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":69398188},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246419456,"data_stored":246415977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25137152,"data_stored":25057135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240893952,"data_stored":240890473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'3829", 'reported_seq': 3900, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.594944+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:18:56.594944+0000', 'last_peered': '2024-09-15T21:18:56.594944+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:18:56.594944+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 3829, 'log_dups_size': 0, 'ondisk_log_size': 3829, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15683584, 'num_objects': 3829, 'num_object_clones': 0, 'num_object_copies': 7658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3829, 'num_write_kb': 15316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 462}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'3905", 'reported_seq': 4911, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:18:58.676341+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:18:58.676341+0000', 'last_peered': '2024-09-15T21:18:58.676341+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:18:58.676341+0000', 'last_undegraded': '2024-09-15T21:18:58.676341+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 3905, 'log_dups_size': 0, 'ondisk_log_size': 3905, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15978496, 'num_objects': 3901, 'num_object_clones': 0, 'num_object_copies': 7802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2947, 'num_objects_unfound': 0, 'num_objects_dirty': 3901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2628, 'num_write_kb': 10512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 445, 'num_bytes_recovered': 1822720, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2947}, {'shards': '2', 'objects': 954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'3876", 'reported_seq': 3997, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.638568+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:18:56.638568+0000', 'last_peered': '2024-09-15T21:18:56.638568+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:18:56.638568+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 3876, 'log_dups_size': 0, 'ondisk_log_size': 3876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15855616, 'num_objects': 3871, 'num_object_clones': 0, 'num_object_copies': 7742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3871, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1301, 'num_write_kb': 5204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 500}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'3943", 'reported_seq': 4061, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.615694+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:18:56.615694+0000', 'last_peered': '2024-09-15T21:18:56.615694+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:18:56.615694+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 3943, 'log_dups_size': 0, 'ondisk_log_size': 3943, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16125952, 'num_objects': 3937, 'num_object_clones': 0, 'num_object_copies': 7874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3937, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1579, 'num_write_kb': 6316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 466}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:00.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'3913", 'reported_seq': 4032, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:58.679809+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:18:58.679809+0000', 'last_peered': '2024-09-15T21:18:58.679809+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:18:58.679809+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 3913, 'log_dups_size': 0, 'ondisk_log_size': 3913, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15998976, 'num_objects': 3906, 'num_object_clones': 0, 'num_object_copies': 7812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2499, 'num_write_kb': 9996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 568}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'3855", 'reported_seq': 3972, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.640615+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:18:56.640615+0000', 'last_peered': '2024-09-15T21:18:56.640615+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:18:56.640615+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 3855, 'log_dups_size': 0, 'ondisk_log_size': 3855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15757312, 'num_objects': 3847, 'num_object_clones': 0, 'num_object_copies': 7694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1381, 'num_write_kb': 5524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 506}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'3882", 'reported_seq': 3986, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:58.699070+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:18:58.699070+0000', 'last_peered': '2024-09-15T21:18:58.699070+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:18:58.699070+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 3882, 'log_dups_size': 0, 'ondisk_log_size': 3882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15880192, 'num_objects': 3877, 'num_object_clones': 0, 'num_object_copies': 7754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2433, 'num_write_kb': 9732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 611}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'3927", 'reported_seq': 4036, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:58.700346+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:18:58.700346+0000', 'last_peered': '2024-09-15T21:18:58.700346+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:18:58.700346+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 3927, 'log_dups_size': 0, 'ondisk_log_size': 3927, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16060416, 'num_objects': 3921, 'num_object_clones': 0, 'num_object_copies': 7842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2326, 'num_write_kb': 9304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 571}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'3924", 'reported_seq': 4047, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.647273+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:18:56.647273+0000', 'last_peered': '2024-09-15T21:18:56.647273+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:18:56.647273+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 3924, 'log_dups_size': 0, 'ondisk_log_size': 3924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16064512, 'num_objects': 3922, 'num_object_clones': 0, 'num_object_copies': 7844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3922, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1231, 'num_write_kb': 4924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 489}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'3908", 'reported_seq': 4032, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.677751+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:18:56.677751+0000', 'last_peered': '2024-09-15T21:18:56.677751+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:18:56.677751+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 3908, 'log_dups_size': 0, 'ondisk_log_size': 3908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15974400, 'num_objects': 3900, 'num_object_clones': 0, 'num_object_copies': 7800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1493, 'num_write_kb': 5972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 521}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:00.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'3856", 'reported_seq': 3970, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.661702+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:18:56.661702+0000', 'last_peered': '2024-09-15T21:18:56.661702+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:18:56.661702+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 3856, 'log_dups_size': 0, 'ondisk_log_size': 3856, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15777792, 'num_objects': 3852, 'num_object_clones': 0, 'num_object_copies': 7704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1503, 'num_write_kb': 6012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 487}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:00.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'3985", 'reported_seq': 4108, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:18:56.665734+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:18:56.665734+0000', 'last_peered': '2024-09-15T21:18:56.665734+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:18:56.665734+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 3985, 'log_dups_size': 0, 'ondisk_log_size': 3985, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16274025, 'num_objects': 3974, 'num_object_clones': 0, 'num_object_copies': 7948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1645, 'num_write_kb': 6580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 504}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:00.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:01.062 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:01.062 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/125108 objects degraded (29.712%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":154,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62554,"data_bytes":256209532,"bytes_used":835608576,"bytes_avail":289074683904,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":125108,"degraded_ratio":0.29711928893436074,"misplaced_objects":2947,"misplaced_total":125108,"misplaced_ratio":0.023555647920196949,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":151768,"recovering_keys_per_sec":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"write_bytes_sec":2469904,"read_op_per_sec":0,"write_op_per_sec":603},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:01.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:01.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:01.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 68 20 64900 64880 3.72634 1.51953 0.0470844 0.0209544 2024-09-15T21:19:01.726 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:01.726 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:01.776 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-15T21:19:00.316971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":256209532,"num_objects":62554,"num_object_clones":0,"num_object_copies":125108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":62554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34049,"num_write_kb":136190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":1822739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62636,"ondisk_log_size":62636,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":816024,"kb_used_data":500264,"kb_used_omap":12,"kb_used_meta":315699,"kb_avail":282299496,"statfs":{"total":289910292480,"available":289074683904,"internally_reserved":0,"allocated":512270336,"data_stored":511427448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":323275786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,5,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29663232,"num_objects":7242,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3393,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7242,"num_write_kb":28968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009869"},"pg_stats":[{"pgid":"2.f","version":"44'3829","reported_seq":3900,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.594944+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:18:56.594944+0000","last_peered":"2024-09-15T21:18:56.594944+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:18:56.594944+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":3829,"log_dups_size":0,"ondisk_log_size":3829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3829,"num_write_kb":15316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":462},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'3968","reported_seq":4059,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:58.676547+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:18:58.676547+0000","last_peered":"2024-09-15T21:18:58.676547+0000","last_clean":"2024-09-15T21:18:58.676547+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:18:58.676547+0000","last_undegraded":"2024-09-15T21:18:58.676547+0000","last_fullsized":"2024-09-15T21:18:58.676547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":3968,"log_dups_size":0,"ondisk_log_size":3968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2694,"num_write_kb":10776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'3905","reported_seq":4911,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:18:58.676341+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:18:58.676341+0000","last_peered":"2024-09-15T21:18:58.676341+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:18:58.676341+0000","last_undegraded":"2024-09-15T21:18:58.676341+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":3905,"log_dups_size":0,"ondisk_log_size":3905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2628,"num_write_kb":10512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2947},{"shards":"2","objects":954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4047","reported_seq":4111,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:58.644785+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:18:58.644785+0000","last_peered":"2024-09-15T21:18:58.644785+0000","last_clean":"2024-09-15T21:18:58.644785+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:18:58.644785+0000","last_undegraded":"2024-09-15T21:18:58.644785+0000","last_fullsized":"2024-09-15T21:18:58.644785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16576512,"num_objects":4047,"num_object_clones":0,"num_object_copies":8094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4047,"num_write_kb":16188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'3908","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.609290+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:18:56.609290+0000","last_peered":"2024-09-15T21:18:56.609290+0000","last_clean":"2024-09-15T21:18:56.609290+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:18:56.609290+0000","last_undegraded":"2024-09-15T21:18:56.609290+0000","last_fullsized":"2024-09-15T21:18:56.609290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'3876","reported_seq":3997,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.638568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:18:56.638568+0000","last_peered":"2024-09-15T21:18:56.638568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:18:56.638568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15855616,"num_objects":3871,"num_object_clones":0,"num_object_copies":7742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1301,"num_write_kb":5204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'3943","reported_seq":4061,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.615694+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:18:56.615694+0000","last_peered":"2024-09-15T21:18:56.615694+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:18:56.615694+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":3943,"log_dups_size":0,"ondisk_log_size":3943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16125952,"num_objects":3937,"num_object_clones":0,"num_object_copies":7874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":466},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'3907","reported_seq":4021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:56.615285+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:18:56.615285+0000","last_peered":"2024-09-15T21:18:56.615285+0000","last_clean":"2024-09-15T21:18:56.615285+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:18:56.615285+0000","last_undegraded":"2024-09-15T21:18:56.615285+0000","last_fullsized":"2024-09-15T21:18:56.615285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":3907,"log_dups_size":0,"ondisk_log_size":3907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15994880,"num_objects":3905,"num_object_clones":0,"num_object_copies":7810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1692,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'3913","reported_seq":4032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.679809+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:18:58.679809+0000","last_peered":"2024-09-15T21:18:58.679809+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:18:58.679809+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":3913,"log_dups_size":0,"ondisk_log_size":3913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'3855","reported_seq":3972,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.640615+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:18:56.640615+0000","last_peered":"2024-09-15T21:18:56.640615+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:18:56.640615+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":3855,"log_dups_size":0,"ondisk_log_size":3855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15757312,"num_objects":3847,"num_object_clones":0,"num_object_copies":7694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1381,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":506},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'3882","reported_seq":3986,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.699070+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:18:58.699070+0000","last_peered":"2024-09-15T21:18:58.699070+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:18:58.699070+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":3882,"log_dups_size":0,"ondisk_log_size":3882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15880192,"num_objects":3877,"num_object_clones":0,"num_object_copies":7754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'3927","reported_seq":4036,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:58.700346+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:18:58.700346+0000","last_peered":"2024-09-15T21:18:58.700346+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:18:58.700346+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":3927,"log_dups_size":0,"ondisk_log_size":3927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2326,"num_write_kb":9304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'3924","reported_seq":4047,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.647273+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:18:56.647273+0000","last_peered":"2024-09-15T21:18:56.647273+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:18:56.647273+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":3924,"log_dups_size":0,"ondisk_log_size":3924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'3908","reported_seq":4032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.677751+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:18:56.677751+0000","last_peered":"2024-09-15T21:18:56.677751+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:18:56.677751+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":3908,"log_dups_size":0,"ondisk_log_size":3908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15974400,"num_objects":3900,"num_object_clones":0,"num_object_copies":7800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":521},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'3856","reported_seq":3970,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.661702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:18:56.661702+0000","last_peered":"2024-09-15T21:18:56.661702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:18:56.661702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":3856,"log_dups_size":0,"ondisk_log_size":3856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1503,"num_write_kb":6012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":487},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'3985","reported_seq":4108,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:18:56.665734+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:18:56.665734+0000","last_peered":"2024-09-15T21:18:56.665734+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:18:56.665734+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16274025,"num_objects":3974,"num_object_clones":0,"num_object_copies":7948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1645,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":504},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":256209513,"num_objects":62552,"num_object_clones":0,"num_object_copies":125104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2947,"num_objects_unfound":0,"num_objects_dirty":62552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34047,"num_write_kb":136188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512450560,"data_stored":512363585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":62633,"ondisk_log_size":62633,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356508,"kb_used_data":234572,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94015332,"statfs":{"total":96636764160,"available":96271699968,"internally_reserved":0,"allocated":240201728,"data_stored":239946278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":124841645},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149720,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366876,"kb_used_data":240844,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94004964,"statfs":{"total":96636764160,"available":96261083136,"internally_reserved":0,"allocated":246624256,"data_stored":246368806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":129035953},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92640,"kb_used_data":24848,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94279200,"statfs":{"total":96636764160,"available":96541900800,"internally_reserved":0,"allocated":25444352,"data_stored":25112364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":69398188},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":246419456,"data_stored":246415977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25137152,"data_stored":25057135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240893952,"data_stored":240890473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:02.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 69 20 65574 65554 3.71049 2.63281 0.00780679 0.021045 2024-09-15T21:19:03.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 70 20 66237 66217 3.69448 2.58984 0.0653017 0.0211351 2024-09-15T21:19:04.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 71 20 66897 66877 3.67876 2.57812 0.0559012 0.021226 2024-09-15T21:19:04.778 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:05.232 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:05.233 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:05.278 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-15T21:19:04.317918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268612220,"num_objects":65582,"num_object_clones":0,"num_object_copies":131164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37077,"num_write_kb":148302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":2355219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65664,"ondisk_log_size":65664,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":879332,"kb_used_data":524724,"kb_used_omap":12,"kb_used_meta":354547,"kb_avail":282236188,"statfs":{"total":289910292480,"available":289009856512,"internally_reserved":0,"allocated":537317376,"data_stored":536474488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":363056138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,10,5,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-416,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008011"},"pg_stats":[{"pgid":"2.f","version":"44'4035","reported_seq":4106,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.731604+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:01.731604+0000","last_peered":"2024-09-15T21:19:01.731604+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:01.731604+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4181","reported_seq":4271,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.734029+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:03.734029+0000","last_peered":"2024-09-15T21:19:03.734029+0000","last_clean":"2024-09-15T21:19:03.734029+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:03.734029+0000","last_undegraded":"2024-09-15T21:19:03.734029+0000","last_fullsized":"2024-09-15T21:19:03.734029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4181,"log_dups_size":0,"ondisk_log_size":4181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2907,"num_write_kb":11628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4117","reported_seq":5383,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:03.693483+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:03.693483+0000","last_peered":"2024-09-15T21:19:03.693483+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:03.693483+0000","last_undegraded":"2024-09-15T21:19:03.693483+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":1296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4221","reported_seq":4285,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.702333+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:03.702333+0000","last_peered":"2024-09-15T21:19:03.702333+0000","last_clean":"2024-09-15T21:19:03.702333+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:03.702333+0000","last_undegraded":"2024-09-15T21:19:03.702333+0000","last_fullsized":"2024-09-15T21:19:03.702333+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4221,"log_dups_size":0,"ondisk_log_size":4221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17289216,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4221,"num_write_kb":16884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4119","reported_seq":4232,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.707804+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:01.707804+0000","last_peered":"2024-09-15T21:19:01.707804+0000","last_clean":"2024-09-15T21:19:01.707804+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:01.707804+0000","last_undegraded":"2024-09-15T21:19:01.707804+0000","last_fullsized":"2024-09-15T21:19:01.707804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4055","reported_seq":4179,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.636290+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:01.636290+0000","last_peered":"2024-09-15T21:19:01.636290+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:01.636290+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4130","reported_seq":4248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.659186+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:01.659186+0000","last_peered":"2024-09-15T21:19:01.659186+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:01.659186+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":653},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4105","reported_seq":4219,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.601111+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:01.601111+0000","last_peered":"2024-09-15T21:19:01.601111+0000","last_clean":"2024-09-15T21:19:01.601111+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:01.601111+0000","last_undegraded":"2024-09-15T21:19:01.601111+0000","last_fullsized":"2024-09-15T21:19:01.601111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4105,"log_dups_size":0,"ondisk_log_size":4105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1890,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4099","reported_seq":4218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.720132+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:03.720132+0000","last_peered":"2024-09-15T21:19:03.720132+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:03.720132+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16760832,"num_objects":4092,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4022","reported_seq":4139,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.665859+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:01.665859+0000","last_peered":"2024-09-15T21:19:01.665859+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:01.665859+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441344,"num_objects":4014,"num_object_clones":0,"num_object_copies":8028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1548,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":673},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4055","reported_seq":4159,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.717549+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:03.717549+0000","last_peered":"2024-09-15T21:19:03.717549+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:03.717549+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":784}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4117","reported_seq":4226,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.708603+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:03.708603+0000","last_peered":"2024-09-15T21:19:03.708603+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:03.708603+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4135","reported_seq":4258,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.670791+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:01.670791+0000","last_peered":"2024-09-15T21:19:01.670791+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:01.670791+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4135,"log_dups_size":0,"ondisk_log_size":4135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16928768,"num_objects":4133,"num_object_clones":0,"num_object_copies":8266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4088","reported_seq":4211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.695958+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:01.695958+0000","last_peered":"2024-09-15T21:19:01.695958+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:01.695958+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16711680,"num_objects":4080,"num_object_clones":0,"num_object_copies":8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1673,"num_write_kb":6692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4034","reported_seq":4151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.702968+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:01.702968+0000","last_peered":"2024-09-15T21:19:01.702968+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:01.702968+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506880,"num_objects":4030,"num_object_clones":0,"num_object_copies":8060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":665},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4148","reported_seq":4270,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.648897+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:01.648897+0000","last_peered":"2024-09-15T21:19:01.648897+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:01.648897+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16941673,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":667},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268612201,"num_objects":65580,"num_object_clones":0,"num_object_copies":131160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37075,"num_write_kb":148300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":537628672,"data_stored":537541697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65661,"ondisk_log_size":65661,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953494,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370952,"kb_used_data":240696,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94000888,"statfs":{"total":96636764160,"available":96256909312,"internally_reserved":0,"allocated":246472704,"data_stored":246217254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":133361325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":249760,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":255754240,"data_stored":255498790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110380,"kb_used_data":34268,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94261460,"statfs":{"total":96636764160,"available":96523735040,"internally_reserved":0,"allocated":35090432,"data_stored":34758444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":255614976,"data_stored":255611497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34926592,"data_stored":34846575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247087104,"data_stored":247083625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:05.280 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:05.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 72 20 67273 67253 3.64807 1.46875 0.0176897 0.0214017 2024-09-15T21:19:06.040 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:06.040 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:06.057 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-15T21:19:04.317918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268612220,"num_objects":65582,"num_object_clones":0,"num_object_copies":131164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37077,"num_write_kb":148302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":2355219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65664,"ondisk_log_size":65664,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":879332,"kb_used_data":524724,"kb_used_omap":12,"kb_used_meta":354547,"kb_avail":282236188,"statfs":{"total":289910292480,"available":289009856512,"internally_reserved":0,"allocated":537317376,"data_stored":536474488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":363056138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,10,5,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-416,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008011"},"pg_stats":[{"pgid":"2.f","version":"44'4035","reported_seq":4106,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.731604+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:01.731604+0000","last_peered":"2024-09-15T21:19:01.731604+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:01.731604+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4181","reported_seq":4271,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.734029+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:03.734029+0000","last_peered":"2024-09-15T21:19:03.734029+0000","last_clean":"2024-09-15T21:19:03.734029+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:03.734029+0000","last_undegraded":"2024-09-15T21:19:03.734029+0000","last_fullsized":"2024-09-15T21:19:03.734029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4181,"log_dups_size":0,"ondisk_log_size":4181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2907,"num_write_kb":11628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4117","reported_seq":5383,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:03.693483+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:03.693483+0000","last_peered":"2024-09-15T21:19:03.693483+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:03.693483+0000","last_undegraded":"2024-09-15T21:19:03.693483+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":1296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4221","reported_seq":4285,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.702333+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:03.702333+0000","last_peered":"2024-09-15T21:19:03.702333+0000","last_clean":"2024-09-15T21:19:03.702333+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:03.702333+0000","last_undegraded":"2024-09-15T21:19:03.702333+0000","last_fullsized":"2024-09-15T21:19:03.702333+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4221,"log_dups_size":0,"ondisk_log_size":4221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17289216,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4221,"num_write_kb":16884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4119","reported_seq":4232,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.707804+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:01.707804+0000","last_peered":"2024-09-15T21:19:01.707804+0000","last_clean":"2024-09-15T21:19:01.707804+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:01.707804+0000","last_undegraded":"2024-09-15T21:19:01.707804+0000","last_fullsized":"2024-09-15T21:19:01.707804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4055","reported_seq":4179,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.636290+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:01.636290+0000","last_peered":"2024-09-15T21:19:01.636290+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:01.636290+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4130","reported_seq":4248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.659186+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:01.659186+0000","last_peered":"2024-09-15T21:19:01.659186+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:01.659186+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":653},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4105","reported_seq":4219,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.601111+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:01.601111+0000","last_peered":"2024-09-15T21:19:01.601111+0000","last_clean":"2024-09-15T21:19:01.601111+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:01.601111+0000","last_undegraded":"2024-09-15T21:19:01.601111+0000","last_fullsized":"2024-09-15T21:19:01.601111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4105,"log_dups_size":0,"ondisk_log_size":4105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1890,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4099","reported_seq":4218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.720132+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:03.720132+0000","last_peered":"2024-09-15T21:19:03.720132+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:03.720132+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16760832,"num_objects":4092,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4022","reported_seq":4139,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.665859+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:01.665859+0000","last_peered":"2024-09-15T21:19:01.665859+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:01.665859+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441344,"num_objects":4014,"num_object_clones":0,"num_object_copies":8028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1548,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":673},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4055","reported_seq":4159,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.717549+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:03.717549+0000","last_peered":"2024-09-15T21:19:03.717549+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:03.717549+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":784}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4117","reported_seq":4226,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.708603+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:03.708603+0000","last_peered":"2024-09-15T21:19:03.708603+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:03.708603+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4135","reported_seq":4258,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.670791+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:01.670791+0000","last_peered":"2024-09-15T21:19:01.670791+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:01.670791+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4135,"log_dups_size":0,"ondisk_log_size":4135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16928768,"num_objects":4133,"num_object_clones":0,"num_object_copies":8266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4088","reported_seq":4211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.695958+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:01.695958+0000","last_peered":"2024-09-15T21:19:01.695958+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:01.695958+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16711680,"num_objects":4080,"num_object_clones":0,"num_object_copies":8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1673,"num_write_kb":6692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4034","reported_seq":4151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.702968+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:01.702968+0000","last_peered":"2024-09-15T21:19:01.702968+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:01.702968+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506880,"num_objects":4030,"num_object_clones":0,"num_object_copies":8060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":665},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4148","reported_seq":4270,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.648897+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:01.648897+0000","last_peered":"2024-09-15T21:19:01.648897+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:01.648897+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16941673,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":667},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268612201,"num_objects":65580,"num_object_clones":0,"num_object_copies":131160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37075,"num_write_kb":148300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":537628672,"data_stored":537541697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65661,"ondisk_log_size":65661,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953494,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370952,"kb_used_data":240696,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94000888,"statfs":{"total":96636764160,"available":96256909312,"internally_reserved":0,"allocated":246472704,"data_stored":246217254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":133361325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":249760,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":255754240,"data_stored":255498790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110380,"kb_used_data":34268,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94261460,"statfs":{"total":96636764160,"available":96523735040,"internally_reserved":0,"allocated":35090432,"data_stored":34758444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":255614976,"data_stored":255611497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34926592,"data_stored":34846575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247087104,"data_stored":247083625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4035", 'reported_seq': 4106, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.731604+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:01.731604+0000', 'last_peered': '2024-09-15T21:19:01.731604+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:01.731604+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4035, 'log_dups_size': 0, 'ondisk_log_size': 4035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16527360, 'num_objects': 4035, 'num_object_clones': 0, 'num_object_copies': 8070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4035, 'num_write_kb': 16140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 668}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4117", 'reported_seq': 5383, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:03.693483+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:03.693483+0000', 'last_peered': '2024-09-15T21:19:03.693483+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:03.693483+0000', 'last_undegraded': '2024-09-15T21:19:03.693483+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4117, 'log_dups_size': 0, 'ondisk_log_size': 4117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16846848, 'num_objects': 4113, 'num_object_clones': 0, 'num_object_copies': 8226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2817, 'num_objects_unfound': 0, 'num_objects_dirty': 4113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2840, 'num_write_kb': 11360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 575, 'num_bytes_recovered': 2355200, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2817}, {'shards': '2', 'objects': 1296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4055", 'reported_seq': 4179, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.636290+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:01.636290+0000', 'last_peered': '2024-09-15T21:19:01.636290+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:01.636290+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4055, 'log_dups_size': 0, 'ondisk_log_size': 4055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16588800, 'num_objects': 4050, 'num_object_clones': 0, 'num_object_copies': 8100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1480, 'num_write_kb': 5920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 679}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4130", 'reported_seq': 4248, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.659186+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:01.659186+0000', 'last_peered': '2024-09-15T21:19:01.659186+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:01.659186+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 4130, 'log_dups_size': 0, 'ondisk_log_size': 4130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16891904, 'num_objects': 4124, 'num_object_clones': 0, 'num_object_copies': 8248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1766, 'num_write_kb': 7064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 653}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:06.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4099", 'reported_seq': 4218, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:03.720132+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:03.720132+0000', 'last_peered': '2024-09-15T21:19:03.720132+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:03.720132+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4099, 'log_dups_size': 0, 'ondisk_log_size': 4099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16760832, 'num_objects': 4092, 'num_object_clones': 0, 'num_object_copies': 8184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2685, 'num_write_kb': 10740, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 754}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4022", 'reported_seq': 4139, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.665859+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:01.665859+0000', 'last_peered': '2024-09-15T21:19:01.665859+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:01.665859+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4022, 'log_dups_size': 0, 'ondisk_log_size': 4022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16441344, 'num_objects': 4014, 'num_object_clones': 0, 'num_object_copies': 8028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1548, 'num_write_kb': 6192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 673}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4055", 'reported_seq': 4159, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:03.717549+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:03.717549+0000', 'last_peered': '2024-09-15T21:19:03.717549+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:03.717549+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4055, 'log_dups_size': 0, 'ondisk_log_size': 4055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16588800, 'num_objects': 4050, 'num_object_clones': 0, 'num_object_copies': 8100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2606, 'num_write_kb': 10424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 784}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4117", 'reported_seq': 4226, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:03.708603+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:03.708603+0000', 'last_peered': '2024-09-15T21:19:03.708603+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:03.708603+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4117, 'log_dups_size': 0, 'ondisk_log_size': 4117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16838656, 'num_objects': 4111, 'num_object_clones': 0, 'num_object_copies': 8222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2516, 'num_write_kb': 10064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 761}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4135", 'reported_seq': 4258, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.670791+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:01.670791+0000', 'last_peered': '2024-09-15T21:19:01.670791+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:01.670791+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 4135, 'log_dups_size': 0, 'ondisk_log_size': 4135, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16928768, 'num_objects': 4133, 'num_object_clones': 0, 'num_object_copies': 8266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1442, 'num_write_kb': 5768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 700}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4088", 'reported_seq': 4211, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.695958+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:01.695958+0000', 'last_peered': '2024-09-15T21:19:01.695958+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:01.695958+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4088, 'log_dups_size': 0, 'ondisk_log_size': 4088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16711680, 'num_objects': 4080, 'num_object_clones': 0, 'num_object_copies': 8160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4080, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1673, 'num_write_kb': 6692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 701}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:06.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4034", 'reported_seq': 4151, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.702968+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:01.702968+0000', 'last_peered': '2024-09-15T21:19:01.702968+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:01.702968+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4034, 'log_dups_size': 0, 'ondisk_log_size': 4034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16506880, 'num_objects': 4030, 'num_object_clones': 0, 'num_object_copies': 8060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1681, 'num_write_kb': 6724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 665}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:06.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4148", 'reported_seq': 4270, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:01.648897+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:01.648897+0000', 'last_peered': '2024-09-15T21:19:01.648897+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:01.648897+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 4148, 'log_dups_size': 0, 'ondisk_log_size': 4148, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16941673, 'num_objects': 4137, 'num_object_clones': 0, 'num_object_copies': 8274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1808, 'num_write_kb': 7232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 667}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:06.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:06.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 73 20 67793 67773 3.62592 2.03125 0.0382625 0.0215355 2024-09-15T21:19:06.955 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:06.955 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/131164 objects degraded (28.340%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":159,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65582,"data_bytes":268612220,"bytes_used":900435968,"bytes_avail":289009856512,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":131164,"degraded_ratio":0.28340093318288556,"misplaced_objects":2817,"misplaced_total":131164,"misplaced_ratio":0.021476929645329511,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97896,"recovering_keys_per_sec":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"write_bytes_sec":2057195,"read_op_per_sec":0,"write_op_per_sec":502},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:07.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:07.022 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:07.468 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:07.481 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 74 20 68299 68279 3.60363 1.97656 0.0045084 0.0216584 2024-09-15T21:19:07.484 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-15T21:19:06.318342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268612220,"num_objects":65582,"num_object_clones":0,"num_object_copies":131164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37077,"num_write_kb":148302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":2355219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65664,"ondisk_log_size":65664,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":879332,"kb_used_data":524724,"kb_used_omap":12,"kb_used_meta":354547,"kb_avail":282236188,"statfs":{"total":289910292480,"available":289009856512,"internally_reserved":0,"allocated":537317376,"data_stored":536474488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":363056138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,10,5,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24702976,"num_objects":6031,"num_object_clones":0,"num_object_copies":12062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":6031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6031,"num_write_kb":24124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008083"},"pg_stats":[{"pgid":"2.f","version":"44'4035","reported_seq":4106,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.731604+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:01.731604+0000","last_peered":"2024-09-15T21:19:01.731604+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:01.731604+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4035,"log_dups_size":0,"ondisk_log_size":4035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4035,"num_write_kb":16140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":668},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4181","reported_seq":4271,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.734029+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:03.734029+0000","last_peered":"2024-09-15T21:19:03.734029+0000","last_clean":"2024-09-15T21:19:03.734029+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:03.734029+0000","last_undegraded":"2024-09-15T21:19:03.734029+0000","last_fullsized":"2024-09-15T21:19:03.734029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4181,"log_dups_size":0,"ondisk_log_size":4181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2907,"num_write_kb":11628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4117","reported_seq":5383,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:03.693483+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:03.693483+0000","last_peered":"2024-09-15T21:19:03.693483+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:03.693483+0000","last_undegraded":"2024-09-15T21:19:03.693483+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":11360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":1296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4221","reported_seq":4285,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:03.702333+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:03.702333+0000","last_peered":"2024-09-15T21:19:03.702333+0000","last_clean":"2024-09-15T21:19:03.702333+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:03.702333+0000","last_undegraded":"2024-09-15T21:19:03.702333+0000","last_fullsized":"2024-09-15T21:19:03.702333+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4221,"log_dups_size":0,"ondisk_log_size":4221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17289216,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4221,"num_write_kb":16884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4119","reported_seq":4232,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.707804+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:01.707804+0000","last_peered":"2024-09-15T21:19:01.707804+0000","last_clean":"2024-09-15T21:19:01.707804+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:01.707804+0000","last_undegraded":"2024-09-15T21:19:01.707804+0000","last_fullsized":"2024-09-15T21:19:01.707804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4055","reported_seq":4179,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.636290+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:01.636290+0000","last_peered":"2024-09-15T21:19:01.636290+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:01.636290+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":679},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4130","reported_seq":4248,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.659186+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:01.659186+0000","last_peered":"2024-09-15T21:19:01.659186+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:01.659186+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16891904,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1766,"num_write_kb":7064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":653},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4105","reported_seq":4219,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:01.601111+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:01.601111+0000","last_peered":"2024-09-15T21:19:01.601111+0000","last_clean":"2024-09-15T21:19:01.601111+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:01.601111+0000","last_undegraded":"2024-09-15T21:19:01.601111+0000","last_fullsized":"2024-09-15T21:19:01.601111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4105,"log_dups_size":0,"ondisk_log_size":4105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1890,"num_write_kb":7560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4099","reported_seq":4218,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.720132+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:03.720132+0000","last_peered":"2024-09-15T21:19:03.720132+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:03.720132+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4099,"log_dups_size":0,"ondisk_log_size":4099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16760832,"num_objects":4092,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4022","reported_seq":4139,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.665859+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:01.665859+0000","last_peered":"2024-09-15T21:19:01.665859+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:01.665859+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4022,"log_dups_size":0,"ondisk_log_size":4022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441344,"num_objects":4014,"num_object_clones":0,"num_object_copies":8028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1548,"num_write_kb":6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":673},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4055","reported_seq":4159,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.717549+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:03.717549+0000","last_peered":"2024-09-15T21:19:03.717549+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:03.717549+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":784}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4117","reported_seq":4226,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:03.708603+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:03.708603+0000","last_peered":"2024-09-15T21:19:03.708603+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:03.708603+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4117,"log_dups_size":0,"ondisk_log_size":4117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4135","reported_seq":4258,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.670791+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:01.670791+0000","last_peered":"2024-09-15T21:19:01.670791+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:01.670791+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4135,"log_dups_size":0,"ondisk_log_size":4135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16928768,"num_objects":4133,"num_object_clones":0,"num_object_copies":8266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":700},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4088","reported_seq":4211,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.695958+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:01.695958+0000","last_peered":"2024-09-15T21:19:01.695958+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:01.695958+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16711680,"num_objects":4080,"num_object_clones":0,"num_object_copies":8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1673,"num_write_kb":6692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4034","reported_seq":4151,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.702968+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:01.702968+0000","last_peered":"2024-09-15T21:19:01.702968+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:01.702968+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506880,"num_objects":4030,"num_object_clones":0,"num_object_copies":8060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1681,"num_write_kb":6724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":665},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4148","reported_seq":4270,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:01.648897+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:01.648897+0000","last_peered":"2024-09-15T21:19:01.648897+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:01.648897+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4148,"log_dups_size":0,"ondisk_log_size":4148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16941673,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":667},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268612201,"num_objects":65580,"num_object_clones":0,"num_object_copies":131160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2817,"num_objects_unfound":0,"num_objects_dirty":65580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37075,"num_write_kb":148300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":575,"num_bytes_recovered":2355200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":537628672,"data_stored":537541697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65661,"ondisk_log_size":65661,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953494,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370952,"kb_used_data":240696,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94000888,"statfs":{"total":96636764160,"available":96256909312,"internally_reserved":0,"allocated":246472704,"data_stored":246217254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":133361325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149721,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398000,"kb_used_data":249760,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93973840,"statfs":{"total":96636764160,"available":96229212160,"internally_reserved":0,"allocated":255754240,"data_stored":255498790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110380,"kb_used_data":34268,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94261460,"statfs":{"total":96636764160,"available":96523735040,"internally_reserved":0,"allocated":35090432,"data_stored":34758444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":77917868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,4,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":255614976,"data_stored":255611497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34926592,"data_stored":34846575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247087104,"data_stored":247083625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:08.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 75 20 68865 68845 3.58506 2.21094 0.0377273 0.0217833 2024-09-15T21:19:09.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 76 20 69526 69506 3.57186 2.58203 0.0267127 0.0218625 2024-09-15T21:19:10.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 77 20 70178 70158 3.55855 2.54688 0.0120498 0.0219443 2024-09-15T21:19:10.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:11.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:11.272 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:11.290 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-15T21:19:10.319231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280056444,"num_objects":68376,"num_object_clones":0,"num_object_copies":136752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":68376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39871,"num_write_kb":159478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68458,"ondisk_log_size":68458,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":940588,"kb_used_data":548472,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282174932,"statfs":{"total":289910292480,"available":288947130368,"internally_reserved":0,"allocated":561635328,"data_stored":560792440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,9,3,3,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28258304,"num_objects":6899,"num_object_clones":0,"num_object_copies":13798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-501,"num_objects_unfound":0,"num_objects_dirty":6899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6899,"num_write_kb":27596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":2052096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002719"},"pg_stats":[{"pgid":"2.f","version":"44'4200","reported_seq":4269,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.756340+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:06.756340+0000","last_peered":"2024-09-15T21:19:06.756340+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:06.756340+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":833},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4358","reported_seq":4449,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.701348+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:08.701348+0000","last_peered":"2024-09-15T21:19:08.701348+0000","last_clean":"2024-09-15T21:19:08.701348+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:08.701348+0000","last_undegraded":"2024-09-15T21:19:08.701348+0000","last_fullsized":"2024-09-15T21:19:08.701348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4272","reported_seq":5966,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:08.739621+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:08.739621+0000","last_peered":"2024-09-15T21:19:08.739621+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:08.739621+0000","last_undegraded":"2024-09-15T21:19:08.739621+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2603},{"shards":"2","objects":1665}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4384","reported_seq":4448,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.696608+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:08.696608+0000","last_peered":"2024-09-15T21:19:08.696608+0000","last_clean":"2024-09-15T21:19:08.696608+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:08.696608+0000","last_undegraded":"2024-09-15T21:19:08.696608+0000","last_fullsized":"2024-09-15T21:19:08.696608+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956864,"num_objects":4384,"num_object_clones":0,"num_object_copies":8768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4384,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4288","reported_seq":4401,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:06.749551+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:06.749551+0000","last_peered":"2024-09-15T21:19:06.749551+0000","last_clean":"2024-09-15T21:19:06.749551+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:06.749551+0000","last_undegraded":"2024-09-15T21:19:06.749551+0000","last_fullsized":"2024-09-15T21:19:06.749551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17534976,"num_objects":4281,"num_object_clones":0,"num_object_copies":8562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":8584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4252","reported_seq":4376,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.753796+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:06.753796+0000","last_peered":"2024-09-15T21:19:06.753796+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:06.753796+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4252,"log_dups_size":0,"ondisk_log_size":4252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4328","reported_seq":4446,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.755895+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:06.755895+0000","last_peered":"2024-09-15T21:19:06.755895+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:06.755895+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702912,"num_objects":4322,"num_object_clones":0,"num_object_copies":8644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1964,"num_write_kb":7856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4273","reported_seq":4387,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:06.774640+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:06.774640+0000","last_peered":"2024-09-15T21:19:06.774640+0000","last_clean":"2024-09-15T21:19:06.774640+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:06.774640+0000","last_undegraded":"2024-09-15T21:19:06.774640+0000","last_fullsized":"2024-09-15T21:19:06.774640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4273,"log_dups_size":0,"ondisk_log_size":4273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2058,"num_write_kb":8232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4269","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.725692+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:08.725692+0000","last_peered":"2024-09-15T21:19:08.725692+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:08.725692+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17457152,"num_objects":4262,"num_object_clones":0,"num_object_copies":8524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2855,"num_write_kb":11420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4215","reported_seq":4332,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.762787+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:06.762787+0000","last_peered":"2024-09-15T21:19:06.762787+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:06.762787+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1741,"num_write_kb":6964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":866},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4206","reported_seq":4309,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.656437+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:08.656437+0000","last_peered":"2024-09-15T21:19:08.656437+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:08.656437+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4206,"log_dups_size":0,"ondisk_log_size":4206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17207296,"num_objects":4201,"num_object_clones":0,"num_object_copies":8402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2757,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4297","reported_seq":4406,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.723077+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:08.723077+0000","last_peered":"2024-09-15T21:19:08.723077+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:08.723077+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2696,"num_write_kb":10784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4311","reported_seq":4434,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.686458+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:06.686458+0000","last_peered":"2024-09-15T21:19:06.686458+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:06.686458+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4311,"log_dups_size":0,"ondisk_log_size":4311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17649664,"num_objects":4309,"num_object_clones":0,"num_object_copies":8618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4247","reported_seq":4371,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.705138+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:06.705138+0000","last_peered":"2024-09-15T21:19:06.705138+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:06.705138+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4247,"log_dups_size":0,"ondisk_log_size":4247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1832,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":860},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4220","reported_seq":4338,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.697141+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:06.697141+0000","last_peered":"2024-09-15T21:19:06.697141+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:06.697141+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4220,"log_dups_size":0,"ondisk_log_size":4220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17268736,"num_objects":4216,"num_object_clones":0,"num_object_copies":8432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1867,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4335","reported_seq":4457,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.763473+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:06.763473+0000","last_peered":"2024-09-15T21:19:06.763473+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:06.763473+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4335,"log_dups_size":0,"ondisk_log_size":4335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707625,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1995,"num_write_kb":7980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280056425,"num_objects":68374,"num_object_clones":0,"num_object_copies":136748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":68374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39869,"num_write_kb":159476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561520640,"data_stored":561433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68455,"ondisk_log_size":68455,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398580,"kb_used_data":246116,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93973260,"statfs":{"total":96636764160,"available":96228618240,"internally_reserved":0,"allocated":252022784,"data_stored":251767334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398568,"kb_used_data":258648,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93973272,"statfs":{"total":96636764160,"available":96228630528,"internally_reserved":0,"allocated":264855552,"data_stored":264600102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":143257265},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,8,0,0,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143440,"kb_used_data":43708,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94228400,"statfs":{"total":96636764160,"available":96489881600,"internally_reserved":0,"allocated":44756992,"data_stored":44425004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":102100652},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264519680,"data_stored":264516201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44556288,"data_stored":44476271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252444672,"data_stored":252441193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:11.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:11.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 78 20 70662 70642 3.53716 1.89062 0.00459585 0.0220059 2024-09-15T21:19:11.611 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:11.629 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-15T21:19:10.319231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280056444,"num_objects":68376,"num_object_clones":0,"num_object_copies":136752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":68376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39871,"num_write_kb":159478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68458,"ondisk_log_size":68458,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":940588,"kb_used_data":548472,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282174932,"statfs":{"total":289910292480,"available":288947130368,"internally_reserved":0,"allocated":561635328,"data_stored":560792440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,9,3,3,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28258304,"num_objects":6899,"num_object_clones":0,"num_object_copies":13798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-501,"num_objects_unfound":0,"num_objects_dirty":6899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6899,"num_write_kb":27596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":2052096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002719"},"pg_stats":[{"pgid":"2.f","version":"44'4200","reported_seq":4269,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.756340+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:06.756340+0000","last_peered":"2024-09-15T21:19:06.756340+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:06.756340+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":833},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4358","reported_seq":4449,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.701348+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:08.701348+0000","last_peered":"2024-09-15T21:19:08.701348+0000","last_clean":"2024-09-15T21:19:08.701348+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:08.701348+0000","last_undegraded":"2024-09-15T21:19:08.701348+0000","last_fullsized":"2024-09-15T21:19:08.701348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4272","reported_seq":5966,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:08.739621+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:08.739621+0000","last_peered":"2024-09-15T21:19:08.739621+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:08.739621+0000","last_undegraded":"2024-09-15T21:19:08.739621+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2603},{"shards":"2","objects":1665}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4384","reported_seq":4448,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.696608+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:08.696608+0000","last_peered":"2024-09-15T21:19:08.696608+0000","last_clean":"2024-09-15T21:19:08.696608+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:08.696608+0000","last_undegraded":"2024-09-15T21:19:08.696608+0000","last_fullsized":"2024-09-15T21:19:08.696608+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956864,"num_objects":4384,"num_object_clones":0,"num_object_copies":8768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4384,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4288","reported_seq":4401,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:06.749551+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:06.749551+0000","last_peered":"2024-09-15T21:19:06.749551+0000","last_clean":"2024-09-15T21:19:06.749551+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:06.749551+0000","last_undegraded":"2024-09-15T21:19:06.749551+0000","last_fullsized":"2024-09-15T21:19:06.749551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17534976,"num_objects":4281,"num_object_clones":0,"num_object_copies":8562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":8584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4252","reported_seq":4376,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.753796+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:06.753796+0000","last_peered":"2024-09-15T21:19:06.753796+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:06.753796+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4252,"log_dups_size":0,"ondisk_log_size":4252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4328","reported_seq":4446,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.755895+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:06.755895+0000","last_peered":"2024-09-15T21:19:06.755895+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:06.755895+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702912,"num_objects":4322,"num_object_clones":0,"num_object_copies":8644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1964,"num_write_kb":7856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4273","reported_seq":4387,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:06.774640+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:06.774640+0000","last_peered":"2024-09-15T21:19:06.774640+0000","last_clean":"2024-09-15T21:19:06.774640+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:06.774640+0000","last_undegraded":"2024-09-15T21:19:06.774640+0000","last_fullsized":"2024-09-15T21:19:06.774640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4273,"log_dups_size":0,"ondisk_log_size":4273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2058,"num_write_kb":8232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4269","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.725692+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:08.725692+0000","last_peered":"2024-09-15T21:19:08.725692+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:08.725692+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17457152,"num_objects":4262,"num_object_clones":0,"num_object_copies":8524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2855,"num_write_kb":11420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4215","reported_seq":4332,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.762787+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:06.762787+0000","last_peered":"2024-09-15T21:19:06.762787+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:06.762787+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1741,"num_write_kb":6964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":866},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4206","reported_seq":4309,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.656437+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:08.656437+0000","last_peered":"2024-09-15T21:19:08.656437+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:08.656437+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4206,"log_dups_size":0,"ondisk_log_size":4206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17207296,"num_objects":4201,"num_object_clones":0,"num_object_copies":8402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2757,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4297","reported_seq":4406,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.723077+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:08.723077+0000","last_peered":"2024-09-15T21:19:08.723077+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:08.723077+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2696,"num_write_kb":10784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4311","reported_seq":4434,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.686458+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:06.686458+0000","last_peered":"2024-09-15T21:19:06.686458+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:06.686458+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4311,"log_dups_size":0,"ondisk_log_size":4311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17649664,"num_objects":4309,"num_object_clones":0,"num_object_copies":8618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4247","reported_seq":4371,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.705138+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:06.705138+0000","last_peered":"2024-09-15T21:19:06.705138+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:06.705138+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4247,"log_dups_size":0,"ondisk_log_size":4247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1832,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":860},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4220","reported_seq":4338,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.697141+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:06.697141+0000","last_peered":"2024-09-15T21:19:06.697141+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:06.697141+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4220,"log_dups_size":0,"ondisk_log_size":4220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17268736,"num_objects":4216,"num_object_clones":0,"num_object_copies":8432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1867,"num_write_kb":7468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4335","reported_seq":4457,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:06.763473+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:06.763473+0000","last_peered":"2024-09-15T21:19:06.763473+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:06.763473+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4335,"log_dups_size":0,"ondisk_log_size":4335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707625,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1995,"num_write_kb":7980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280056425,"num_objects":68374,"num_object_clones":0,"num_object_copies":136748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":68374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39869,"num_write_kb":159476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561520640,"data_stored":561433665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68455,"ondisk_log_size":68455,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398580,"kb_used_data":246116,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93973260,"statfs":{"total":96636764160,"available":96228618240,"internally_reserved":0,"allocated":252022784,"data_stored":251767334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398568,"kb_used_data":258648,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93973272,"statfs":{"total":96636764160,"available":96228630528,"internally_reserved":0,"allocated":264855552,"data_stored":264600102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":143257265},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,8,0,0,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143440,"kb_used_data":43708,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94228400,"statfs":{"total":96636764160,"available":96489881600,"internally_reserved":0,"allocated":44756992,"data_stored":44425004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":102100652},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264519680,"data_stored":264516201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44556288,"data_stored":44476271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252444672,"data_stored":252441193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4200", 'reported_seq': 4269, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.756340+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:06.756340+0000', 'last_peered': '2024-09-15T21:19:06.756340+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:06.756340+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4200, 'log_dups_size': 0, 'ondisk_log_size': 4200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17203200, 'num_objects': 4200, 'num_object_clones': 0, 'num_object_copies': 8400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4200, 'num_write_kb': 16800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 833}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4272", 'reported_seq': 5966, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:08.739621+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:08.739621+0000', 'last_peered': '2024-09-15T21:19:08.739621+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:08.739621+0000', 'last_undegraded': '2024-09-15T21:19:08.739621+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4272, 'log_dups_size': 0, 'ondisk_log_size': 4272, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17481728, 'num_objects': 4268, 'num_object_clones': 0, 'num_object_copies': 8536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2603, 'num_objects_unfound': 0, 'num_objects_dirty': 4268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2995, 'num_write_kb': 11980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 789, 'num_bytes_recovered': 3231744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2603}, {'shards': '2', 'objects': 1665}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4252", 'reported_seq': 4376, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.753796+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:06.753796+0000', 'last_peered': '2024-09-15T21:19:06.753796+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:06.753796+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4252, 'log_dups_size': 0, 'ondisk_log_size': 4252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17395712, 'num_objects': 4247, 'num_object_clones': 0, 'num_object_copies': 8494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1677, 'num_write_kb': 6708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 876}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.631 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:11.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4328", 'reported_seq': 4446, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.755895+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:06.755895+0000', 'last_peered': '2024-09-15T21:19:06.755895+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:06.755895+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 4328, 'log_dups_size': 0, 'ondisk_log_size': 4328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17702912, 'num_objects': 4322, 'num_object_clones': 0, 'num_object_copies': 8644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1964, 'num_write_kb': 7856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 851}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:11.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4269", 'reported_seq': 4388, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:08.725692+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:08.725692+0000', 'last_peered': '2024-09-15T21:19:08.725692+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:08.725692+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4269, 'log_dups_size': 0, 'ondisk_log_size': 4269, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17457152, 'num_objects': 4262, 'num_object_clones': 0, 'num_object_copies': 8524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2855, 'num_write_kb': 11420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 924}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:11.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:11.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4215", 'reported_seq': 4332, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.762787+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:06.762787+0000', 'last_peered': '2024-09-15T21:19:06.762787+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:06.762787+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4215, 'log_dups_size': 0, 'ondisk_log_size': 4215, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17231872, 'num_objects': 4207, 'num_object_clones': 0, 'num_object_copies': 8414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1741, 'num_write_kb': 6964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 866}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4206", 'reported_seq': 4309, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:08.656437+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:08.656437+0000', 'last_peered': '2024-09-15T21:19:08.656437+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:08.656437+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4206, 'log_dups_size': 0, 'ondisk_log_size': 4206, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17207296, 'num_objects': 4201, 'num_object_clones': 0, 'num_object_copies': 8402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2757, 'num_write_kb': 11028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 935}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4297", 'reported_seq': 4406, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:08.723077+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:08.723077+0000', 'last_peered': '2024-09-15T21:19:08.723077+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:08.723077+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4297, 'log_dups_size': 0, 'ondisk_log_size': 4297, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17575936, 'num_objects': 4291, 'num_object_clones': 0, 'num_object_copies': 8582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2696, 'num_write_kb': 10784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 941}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4311", 'reported_seq': 4434, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.686458+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:06.686458+0000', 'last_peered': '2024-09-15T21:19:06.686458+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:06.686458+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 4311, 'log_dups_size': 0, 'ondisk_log_size': 4311, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17649664, 'num_objects': 4309, 'num_object_clones': 0, 'num_object_copies': 8618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1618, 'num_write_kb': 6472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 876}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4247", 'reported_seq': 4371, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.705138+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:06.705138+0000', 'last_peered': '2024-09-15T21:19:06.705138+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:06.705138+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4247, 'log_dups_size': 0, 'ondisk_log_size': 4247, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17362944, 'num_objects': 4239, 'num_object_clones': 0, 'num_object_copies': 8478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1832, 'num_write_kb': 7328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 860}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.633 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4220", 'reported_seq': 4338, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.697141+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:06.697141+0000', 'last_peered': '2024-09-15T21:19:06.697141+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:06.697141+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4220, 'log_dups_size': 0, 'ondisk_log_size': 4220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17268736, 'num_objects': 4216, 'num_object_clones': 0, 'num_object_copies': 8432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1867, 'num_write_kb': 7468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 851}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4335", 'reported_seq': 4457, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:06.763473+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:06.763473+0000', 'last_peered': '2024-09-15T21:19:06.763473+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:06.763473+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 4335, 'log_dups_size': 0, 'ondisk_log_size': 4335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17707625, 'num_objects': 4324, 'num_object_clones': 0, 'num_object_copies': 8648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1995, 'num_write_kb': 7980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 854}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:11.634 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:12.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:12.024 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/136752 objects degraded (27.182%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68376,"data_bytes":280056444,"bytes_used":963162112,"bytes_avail":288947130368,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":136752,"degraded_ratio":0.2718205218205218,"misplaced_objects":2603,"misplaced_total":136752,"misplaced_ratio":0.019034456534456533,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":170969,"recovering_keys_per_sec":0,"num_objects_recovered":501,"num_bytes_recovered":2052096,"num_keys_recovered":0,"write_bytes_sec":2354325,"read_op_per_sec":0,"write_op_per_sec":574},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:12.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:12.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:12.326 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:12.326 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:12.342 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-15T21:19:12.319746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286835324,"num_objects":70031,"num_object_clones":0,"num_object_copies":140062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":70031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41526,"num_write_kb":166098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":790,"num_bytes_recovered":3231763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70113,"ondisk_log_size":70113,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":968976,"kb_used_data":564384,"kb_used_omap":12,"kb_used_meta":404531,"kb_avail":282146544,"statfs":{"total":289910292480,"available":288918061056,"internally_reserved":0,"allocated":577929216,"data_stored":577086328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":414239754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,5,8,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30625792,"num_objects":7477,"num_object_clones":0,"num_object_copies":14954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":7477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7477,"num_write_kb":29908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002775"},"pg_stats":[{"pgid":"2.f","version":"44'4372","reported_seq":4443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.231725+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:11.231725+0000","last_peered":"2024-09-15T21:19:11.231725+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:11.231725+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4372,"num_write_kb":17488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4358","reported_seq":4449,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.701348+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:08.701348+0000","last_peered":"2024-09-15T21:19:08.701348+0000","last_clean":"2024-09-15T21:19:08.701348+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:08.701348+0000","last_undegraded":"2024-09-15T21:19:08.701348+0000","last_fullsized":"2024-09-15T21:19:08.701348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4272","reported_seq":5966,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:08.739621+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:08.739621+0000","last_peered":"2024-09-15T21:19:08.739621+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:08.739621+0000","last_undegraded":"2024-09-15T21:19:08.739621+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2603},{"shards":"2","objects":1665}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4384","reported_seq":4448,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:08.696608+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:08.696608+0000","last_peered":"2024-09-15T21:19:08.696608+0000","last_clean":"2024-09-15T21:19:08.696608+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:08.696608+0000","last_undegraded":"2024-09-15T21:19:08.696608+0000","last_fullsized":"2024-09-15T21:19:08.696608+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956864,"num_objects":4384,"num_object_clones":0,"num_object_copies":8768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4384,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4448","reported_seq":4561,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.219914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:11.219914+0000","last_peered":"2024-09-15T21:19:11.219914+0000","last_clean":"2024-09-15T21:19:11.219914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:11.219914+0000","last_undegraded":"2024-09-15T21:19:11.219914+0000","last_fullsized":"2024-09-15T21:19:11.219914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190336,"num_objects":4441,"num_object_clones":0,"num_object_copies":8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4436","reported_seq":4560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.244089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:11.244089+0000","last_peered":"2024-09-15T21:19:11.244089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:11.244089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4501","reported_seq":4619,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.255963+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:11.255963+0000","last_peered":"2024-09-15T21:19:11.255963+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:11.255963+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2137,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4413","reported_seq":4527,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.198680+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:11.198680+0000","last_peered":"2024-09-15T21:19:11.198680+0000","last_clean":"2024-09-15T21:19:11.198680+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:11.198680+0000","last_undegraded":"2024-09-15T21:19:11.198680+0000","last_fullsized":"2024-09-15T21:19:11.198680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4269","reported_seq":4388,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.725692+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:08.725692+0000","last_peered":"2024-09-15T21:19:08.725692+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:08.725692+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17457152,"num_objects":4262,"num_object_clones":0,"num_object_copies":8524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2855,"num_write_kb":11420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4377","reported_seq":4494,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.224722+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:11.224722+0000","last_peered":"2024-09-15T21:19:11.224722+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:11.224722+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17895424,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1903,"num_write_kb":7612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4206","reported_seq":4309,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.656437+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:08.656437+0000","last_peered":"2024-09-15T21:19:08.656437+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:08.656437+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4206,"log_dups_size":0,"ondisk_log_size":4206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17207296,"num_objects":4201,"num_object_clones":0,"num_object_copies":8402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2757,"num_write_kb":11028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4297","reported_seq":4406,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:08.723077+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:08.723077+0000","last_peered":"2024-09-15T21:19:08.723077+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:08.723077+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2696,"num_write_kb":10784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4598,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.216582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:11.216582+0000","last_peered":"2024-09-15T21:19:11.216582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:11.216582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1782,"num_write_kb":7128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1040},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4416","reported_seq":4540,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.242597+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:11.242597+0000","last_peered":"2024-09-15T21:19:11.242597+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:11.242597+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055168,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1029},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4389","reported_seq":4507,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.250665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:11.250665+0000","last_peered":"2024-09-15T21:19:11.250665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:11.250665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1020},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4497","reported_seq":4620,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.142529+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:11.142529+0000","last_peered":"2024-09-15T21:19:11.142529+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:11.142529+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371177,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":8628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1016},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286835305,"num_objects":70029,"num_object_clones":0,"num_object_copies":140058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2603,"num_objects_unfound":0,"num_objects_dirty":70029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41524,"num_write_kb":166096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":3231744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":579186688,"data_stored":579099713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70110,"ondisk_log_size":70110,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953495,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398580,"kb_used_data":246116,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93973260,"statfs":{"total":96636764160,"available":96228618240,"internally_reserved":0,"allocated":252022784,"data_stored":251767334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410592,"kb_used_data":266512,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93961248,"statfs":{"total":96636764160,"available":96216317952,"internally_reserved":0,"allocated":272908288,"data_stored":272652838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":147517105},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159804,"kb_used_data":51756,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94212036,"statfs":{"total":96636764160,"available":96473124864,"internally_reserved":0,"allocated":52998144,"data_stored":52666156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":110620332},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272535552,"data_stored":272532073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54206464,"data_stored":54126447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252444672,"data_stored":252441193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:12.482 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 79 20 70726 70706 3.49555 0.25 0.0056764 0.0220938 2024-09-15T21:19:13.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:19:13.481236+0000 min lat: 0.00128075 max lat: 2.56993 avg lat: 0.0220938 2024-09-15T21:19:13.600 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:19:13.601 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 80 20 70726 70706 3.45186 0 - 0.0220938 2024-09-15T21:19:14.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 81 20 70726 70706 3.40924 0 - 0.0220938 2024-09-15T21:19:15.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:15.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 82 20 70747 70727 3.36867 0.0273438 3.57135 0.0230254 2024-09-15T21:19:15.919 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:15.919 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:15.935 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-15T21:19:14.320534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289423996,"num_objects":70663,"num_object_clones":0,"num_object_copies":141326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42158,"num_write_kb":168626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":3645459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70745,"ondisk_log_size":70745,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":960620,"kb_used_data":568508,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282154900,"statfs":{"total":289910292480,"available":288926617600,"internally_reserved":0,"allocated":582152192,"data_stored":581309304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,6,3,4,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25513984,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-445,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":1822720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003103"},"pg_stats":[{"pgid":"2.f","version":"44'4372","reported_seq":4443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.231725+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:11.231725+0000","last_peered":"2024-09-15T21:19:11.231725+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:11.231725+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4372,"num_write_kb":17488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4457","reported_seq":4548,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.861606+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:11.861606+0000","last_peered":"2024-09-15T21:19:11.861606+0000","last_clean":"2024-09-15T21:19:11.861606+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:11.861606+0000","last_undegraded":"2024-09-15T21:19:11.861606+0000","last_fullsized":"2024-09-15T21:19:11.861606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4389","reported_seq":6285,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:11.643145+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:11.643145+0000","last_peered":"2024-09-15T21:19:11.643145+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:11.643145+0000","last_undegraded":"2024-09-15T21:19:11.643145+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2502},{"shards":"2","objects":1883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4482","reported_seq":4546,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.887085+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:11.887085+0000","last_peered":"2024-09-15T21:19:11.887085+0000","last_clean":"2024-09-15T21:19:11.887085+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:11.887085+0000","last_undegraded":"2024-09-15T21:19:11.887085+0000","last_fullsized":"2024-09-15T21:19:11.887085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358272,"num_objects":4482,"num_object_clones":0,"num_object_copies":8964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4448","reported_seq":4561,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.219914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:11.219914+0000","last_peered":"2024-09-15T21:19:11.219914+0000","last_clean":"2024-09-15T21:19:11.219914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:11.219914+0000","last_undegraded":"2024-09-15T21:19:11.219914+0000","last_fullsized":"2024-09-15T21:19:11.219914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190336,"num_objects":4441,"num_object_clones":0,"num_object_copies":8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4436","reported_seq":4560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.244089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:11.244089+0000","last_peered":"2024-09-15T21:19:11.244089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:11.244089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4501","reported_seq":4619,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.255963+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:11.255963+0000","last_peered":"2024-09-15T21:19:11.255963+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:11.255963+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2137,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4413","reported_seq":4527,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.198680+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:11.198680+0000","last_peered":"2024-09-15T21:19:11.198680+0000","last_clean":"2024-09-15T21:19:11.198680+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:11.198680+0000","last_undegraded":"2024-09-15T21:19:11.198680+0000","last_fullsized":"2024-09-15T21:19:11.198680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4359","reported_seq":4478,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.124145+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:11.124145+0000","last_peered":"2024-09-15T21:19:11.124145+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:11.124145+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4359,"log_dups_size":0,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4377","reported_seq":4494,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.224722+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:11.224722+0000","last_peered":"2024-09-15T21:19:11.224722+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:11.224722+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17895424,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1903,"num_write_kb":7612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4318","reported_seq":4422,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.202791+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:11.202791+0000","last_peered":"2024-09-15T21:19:11.202791+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:11.202791+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4318,"log_dups_size":0,"ondisk_log_size":4318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4413","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.174421+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:11.174421+0000","last_peered":"2024-09-15T21:19:11.174421+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:11.174421+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2812,"num_write_kb":11248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4598,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.216582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:11.216582+0000","last_peered":"2024-09-15T21:19:11.216582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:11.216582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1782,"num_write_kb":7128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1040},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4416","reported_seq":4540,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.242597+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:11.242597+0000","last_peered":"2024-09-15T21:19:11.242597+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:11.242597+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055168,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1029},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4389","reported_seq":4507,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.250665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:11.250665+0000","last_peered":"2024-09-15T21:19:11.250665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:11.250665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1020},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4497","reported_seq":4620,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.142529+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:11.142529+0000","last_peered":"2024-09-15T21:19:11.142529+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:11.142529+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371177,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":8628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1016},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289423977,"num_objects":70661,"num_object_clones":0,"num_object_copies":141322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42156,"num_write_kb":168624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582615040,"data_stored":582528065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70742,"ondisk_log_size":70742,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":390224,"kb_used_data":250240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93981616,"statfs":{"total":96636764160,"available":96237174784,"internally_reserved":0,"allocated":256245760,"data_stored":255990310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410592,"kb_used_data":266512,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93961248,"statfs":{"total":96636764160,"available":96216317952,"internally_reserved":0,"allocated":272908288,"data_stored":272652838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":147517105},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159804,"kb_used_data":51756,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94212036,"statfs":{"total":96636764160,"available":96473124864,"internally_reserved":0,"allocated":52998144,"data_stored":52666156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":110620332},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272535552,"data_stored":272532073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54206464,"data_stored":54126447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255873024,"data_stored":255869545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:15.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:16.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 83 20 71263 71243 3.35237 2.01562 0.0166475 0.0232919 2024-09-15T21:19:16.566 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:16.567 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:16.586 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-15T21:19:16.322612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289423996,"num_objects":70663,"num_object_clones":0,"num_object_copies":141326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42158,"num_write_kb":168626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":3645459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70745,"ondisk_log_size":70745,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":960620,"kb_used_data":568508,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282154900,"statfs":{"total":289910292480,"available":288926617600,"internally_reserved":0,"allocated":582152192,"data_stored":581309304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,6,3,4,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-315,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5081,"num_write_kb":20324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":1290240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004694"},"pg_stats":[{"pgid":"2.f","version":"44'4372","reported_seq":4443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.231725+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:11.231725+0000","last_peered":"2024-09-15T21:19:11.231725+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:11.231725+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4372,"num_write_kb":17488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4457","reported_seq":4548,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.861606+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:11.861606+0000","last_peered":"2024-09-15T21:19:11.861606+0000","last_clean":"2024-09-15T21:19:11.861606+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:11.861606+0000","last_undegraded":"2024-09-15T21:19:11.861606+0000","last_fullsized":"2024-09-15T21:19:11.861606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4389","reported_seq":6285,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:11.643145+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:11.643145+0000","last_peered":"2024-09-15T21:19:11.643145+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:11.643145+0000","last_undegraded":"2024-09-15T21:19:11.643145+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2502},{"shards":"2","objects":1883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4482","reported_seq":4546,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.887085+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:11.887085+0000","last_peered":"2024-09-15T21:19:11.887085+0000","last_clean":"2024-09-15T21:19:11.887085+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:11.887085+0000","last_undegraded":"2024-09-15T21:19:11.887085+0000","last_fullsized":"2024-09-15T21:19:11.887085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358272,"num_objects":4482,"num_object_clones":0,"num_object_copies":8964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4448","reported_seq":4561,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.219914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:11.219914+0000","last_peered":"2024-09-15T21:19:11.219914+0000","last_clean":"2024-09-15T21:19:11.219914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:11.219914+0000","last_undegraded":"2024-09-15T21:19:11.219914+0000","last_fullsized":"2024-09-15T21:19:11.219914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190336,"num_objects":4441,"num_object_clones":0,"num_object_copies":8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4436","reported_seq":4560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.244089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:11.244089+0000","last_peered":"2024-09-15T21:19:11.244089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:11.244089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4501","reported_seq":4619,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.255963+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:11.255963+0000","last_peered":"2024-09-15T21:19:11.255963+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:11.255963+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2137,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4413","reported_seq":4527,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.198680+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:11.198680+0000","last_peered":"2024-09-15T21:19:11.198680+0000","last_clean":"2024-09-15T21:19:11.198680+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:11.198680+0000","last_undegraded":"2024-09-15T21:19:11.198680+0000","last_fullsized":"2024-09-15T21:19:11.198680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4359","reported_seq":4478,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.124145+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:11.124145+0000","last_peered":"2024-09-15T21:19:11.124145+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:11.124145+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4359,"log_dups_size":0,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4377","reported_seq":4494,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.224722+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:11.224722+0000","last_peered":"2024-09-15T21:19:11.224722+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:11.224722+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17895424,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1903,"num_write_kb":7612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4318","reported_seq":4422,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.202791+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:11.202791+0000","last_peered":"2024-09-15T21:19:11.202791+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:11.202791+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4318,"log_dups_size":0,"ondisk_log_size":4318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4413","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.174421+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:11.174421+0000","last_peered":"2024-09-15T21:19:11.174421+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:11.174421+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2812,"num_write_kb":11248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4598,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.216582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:11.216582+0000","last_peered":"2024-09-15T21:19:11.216582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:11.216582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1782,"num_write_kb":7128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1040},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4416","reported_seq":4540,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.242597+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:11.242597+0000","last_peered":"2024-09-15T21:19:11.242597+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:11.242597+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055168,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1029},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4389","reported_seq":4507,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.250665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:11.250665+0000","last_peered":"2024-09-15T21:19:11.250665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:11.250665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1020},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4497","reported_seq":4620,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.142529+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:11.142529+0000","last_peered":"2024-09-15T21:19:11.142529+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:11.142529+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371177,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":8628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1016},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289423977,"num_objects":70661,"num_object_clones":0,"num_object_copies":141322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42156,"num_write_kb":168624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582615040,"data_stored":582528065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70742,"ondisk_log_size":70742,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":390224,"kb_used_data":250240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93981616,"statfs":{"total":96636764160,"available":96237174784,"internally_reserved":0,"allocated":256245760,"data_stored":255990310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410592,"kb_used_data":266512,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93961248,"statfs":{"total":96636764160,"available":96216317952,"internally_reserved":0,"allocated":272908288,"data_stored":272652838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":147517105},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159804,"kb_used_data":51756,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94212036,"statfs":{"total":96636764160,"available":96473124864,"internally_reserved":0,"allocated":52998144,"data_stored":52666156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":110620332},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272535552,"data_stored":272532073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54206464,"data_stored":54126447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255873024,"data_stored":255869545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4372", 'reported_seq': 4443, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.231725+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:11.231725+0000', 'last_peered': '2024-09-15T21:19:11.231725+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:11.231725+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4372, 'log_dups_size': 0, 'ondisk_log_size': 4372, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17907712, 'num_objects': 4372, 'num_object_clones': 0, 'num_object_copies': 8744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4372, 'num_write_kb': 17488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1005}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4389", 'reported_seq': 6285, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:11.643145+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:11.643145+0000', 'last_peered': '2024-09-15T21:19:11.643145+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:11.643145+0000', 'last_undegraded': '2024-09-15T21:19:11.643145+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4389, 'log_dups_size': 0, 'ondisk_log_size': 4389, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17960960, 'num_objects': 4385, 'num_object_clones': 0, 'num_object_copies': 8770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2502, 'num_objects_unfound': 0, 'num_objects_dirty': 4385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3112, 'num_write_kb': 12448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 890, 'num_bytes_recovered': 3645440, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2502}, {'shards': '2', 'objects': 1883}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4436", 'reported_seq': 4560, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.244089+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:11.244089+0000', 'last_peered': '2024-09-15T21:19:11.244089+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:11.244089+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4436, 'log_dups_size': 0, 'ondisk_log_size': 4436, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18149376, 'num_objects': 4431, 'num_object_clones': 0, 'num_object_copies': 8862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1861, 'num_write_kb': 7444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1060}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4501", 'reported_seq': 4619, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.255963+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:11.255963+0000', 'last_peered': '2024-09-15T21:19:11.255963+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:11.255963+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 4501, 'log_dups_size': 0, 'ondisk_log_size': 4501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18411520, 'num_objects': 4495, 'num_object_clones': 0, 'num_object_copies': 8990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2137, 'num_write_kb': 8548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1024}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:16.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4359", 'reported_seq': 4478, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.124145+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:11.124145+0000', 'last_peered': '2024-09-15T21:19:11.124145+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:11.124145+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4359, 'log_dups_size': 0, 'ondisk_log_size': 4359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17825792, 'num_objects': 4352, 'num_object_clones': 0, 'num_object_copies': 8704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2945, 'num_write_kb': 11780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4377", 'reported_seq': 4494, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.224722+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:11.224722+0000', 'last_peered': '2024-09-15T21:19:11.224722+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:11.224722+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4377, 'log_dups_size': 0, 'ondisk_log_size': 4377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17895424, 'num_objects': 4369, 'num_object_clones': 0, 'num_object_copies': 8738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1903, 'num_write_kb': 7612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1028}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4318", 'reported_seq': 4422, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.202791+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:11.202791+0000', 'last_peered': '2024-09-15T21:19:11.202791+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:11.202791+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4318, 'log_dups_size': 0, 'ondisk_log_size': 4318, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17666048, 'num_objects': 4313, 'num_object_clones': 0, 'num_object_copies': 8626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2869, 'num_write_kb': 11476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1047}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4413", 'reported_seq': 4522, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.174421+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:11.174421+0000', 'last_peered': '2024-09-15T21:19:11.174421+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:11.174421+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4413, 'log_dups_size': 0, 'ondisk_log_size': 4413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18051072, 'num_objects': 4407, 'num_object_clones': 0, 'num_object_copies': 8814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2812, 'num_write_kb': 11248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1057}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:16.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4475", 'reported_seq': 4598, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.216582+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:11.216582+0000', 'last_peered': '2024-09-15T21:19:11.216582+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:11.216582+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 4475, 'log_dups_size': 0, 'ondisk_log_size': 4475, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18321408, 'num_objects': 4473, 'num_object_clones': 0, 'num_object_copies': 8946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1782, 'num_write_kb': 7128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1040}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4416", 'reported_seq': 4540, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.242597+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:11.242597+0000', 'last_peered': '2024-09-15T21:19:11.242597+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:11.242597+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4416, 'log_dups_size': 0, 'ondisk_log_size': 4416, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18055168, 'num_objects': 4408, 'num_object_clones': 0, 'num_object_copies': 8816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2001, 'num_write_kb': 8004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1029}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4389", 'reported_seq': 4507, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.250665+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:11.250665+0000', 'last_peered': '2024-09-15T21:19:11.250665+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:11.250665+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4389, 'log_dups_size': 0, 'ondisk_log_size': 4389, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17960960, 'num_objects': 4385, 'num_object_clones': 0, 'num_object_copies': 8770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2036, 'num_write_kb': 8144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1020}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:16.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4497", 'reported_seq': 4620, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:11.142529+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:11.142529+0000', 'last_peered': '2024-09-15T21:19:11.142529+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:11.142529+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 4497, 'log_dups_size': 0, 'ondisk_log_size': 4497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18371177, 'num_objects': 4486, 'num_object_clones': 0, 'num_object_copies': 8972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2157, 'num_write_kb': 8628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1016}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:16.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:17.283 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:17.283 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/141326 objects degraded (26.302%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":170,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70663,"data_bytes":289423996,"bytes_used":983674880,"bytes_avail":288926617600,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":141326,"degraded_ratio":0.26302308138629832,"misplaced_objects":2502,"misplaced_total":141326,"misplaced_ratio":0.017703748779417799,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":107477,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":1290240,"num_keys_recovered":0,"write_bytes_sec":1733636,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:17.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:17.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:17.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 84 20 71759 71739 3.33552 1.9375 0.0738766 0.0234123 2024-09-15T21:19:18.128 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:18.128 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:18.213 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-15T21:19:16.322612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289423996,"num_objects":70663,"num_object_clones":0,"num_object_copies":141326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42158,"num_write_kb":168626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":3645459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70745,"ondisk_log_size":70745,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":960620,"kb_used_data":568508,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282154900,"statfs":{"total":289910292480,"available":288926617600,"internally_reserved":0,"allocated":582152192,"data_stored":581309304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,6,3,4,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-315,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5081,"num_write_kb":20324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":1290240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004694"},"pg_stats":[{"pgid":"2.f","version":"44'4372","reported_seq":4443,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.231725+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:11.231725+0000","last_peered":"2024-09-15T21:19:11.231725+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:11.231725+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907712,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4372,"num_write_kb":17488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1005},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4457","reported_seq":4548,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.861606+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:11.861606+0000","last_peered":"2024-09-15T21:19:11.861606+0000","last_clean":"2024-09-15T21:19:11.861606+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:11.861606+0000","last_undegraded":"2024-09-15T21:19:11.861606+0000","last_fullsized":"2024-09-15T21:19:11.861606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4389","reported_seq":6285,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:11.643145+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:11.643145+0000","last_peered":"2024-09-15T21:19:11.643145+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:11.643145+0000","last_undegraded":"2024-09-15T21:19:11.643145+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2502},{"shards":"2","objects":1883}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4482","reported_seq":4546,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.887085+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:11.887085+0000","last_peered":"2024-09-15T21:19:11.887085+0000","last_clean":"2024-09-15T21:19:11.887085+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:11.887085+0000","last_undegraded":"2024-09-15T21:19:11.887085+0000","last_fullsized":"2024-09-15T21:19:11.887085+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358272,"num_objects":4482,"num_object_clones":0,"num_object_copies":8964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4448","reported_seq":4561,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.219914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:11.219914+0000","last_peered":"2024-09-15T21:19:11.219914+0000","last_clean":"2024-09-15T21:19:11.219914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:11.219914+0000","last_undegraded":"2024-09-15T21:19:11.219914+0000","last_fullsized":"2024-09-15T21:19:11.219914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4448,"log_dups_size":0,"ondisk_log_size":4448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190336,"num_objects":4441,"num_object_clones":0,"num_object_copies":8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4436","reported_seq":4560,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.244089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:11.244089+0000","last_peered":"2024-09-15T21:19:11.244089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:11.244089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1060},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4501","reported_seq":4619,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.255963+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:11.255963+0000","last_peered":"2024-09-15T21:19:11.255963+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:11.255963+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18411520,"num_objects":4495,"num_object_clones":0,"num_object_copies":8990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2137,"num_write_kb":8548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4413","reported_seq":4527,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:11.198680+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:11.198680+0000","last_peered":"2024-09-15T21:19:11.198680+0000","last_clean":"2024-09-15T21:19:11.198680+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:11.198680+0000","last_undegraded":"2024-09-15T21:19:11.198680+0000","last_fullsized":"2024-09-15T21:19:11.198680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18067456,"num_objects":4411,"num_object_clones":0,"num_object_copies":8822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4359","reported_seq":4478,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.124145+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:11.124145+0000","last_peered":"2024-09-15T21:19:11.124145+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:11.124145+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4359,"log_dups_size":0,"ondisk_log_size":4359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1014}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4377","reported_seq":4494,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.224722+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:11.224722+0000","last_peered":"2024-09-15T21:19:11.224722+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:11.224722+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17895424,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1903,"num_write_kb":7612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4318","reported_seq":4422,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.202791+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:11.202791+0000","last_peered":"2024-09-15T21:19:11.202791+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:11.202791+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4318,"log_dups_size":0,"ondisk_log_size":4318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4413","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.174421+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:11.174421+0000","last_peered":"2024-09-15T21:19:11.174421+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:11.174421+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4413,"log_dups_size":0,"ondisk_log_size":4413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051072,"num_objects":4407,"num_object_clones":0,"num_object_copies":8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2812,"num_write_kb":11248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4475","reported_seq":4598,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.216582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:11.216582+0000","last_peered":"2024-09-15T21:19:11.216582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:11.216582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4475,"log_dups_size":0,"ondisk_log_size":4475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1782,"num_write_kb":7128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1040},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4416","reported_seq":4540,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.242597+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:11.242597+0000","last_peered":"2024-09-15T21:19:11.242597+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:11.242597+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18055168,"num_objects":4408,"num_object_clones":0,"num_object_copies":8816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1029},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4389","reported_seq":4507,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.250665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:11.250665+0000","last_peered":"2024-09-15T21:19:11.250665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:11.250665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4389,"log_dups_size":0,"ondisk_log_size":4389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17960960,"num_objects":4385,"num_object_clones":0,"num_object_copies":8770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2036,"num_write_kb":8144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1020},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4497","reported_seq":4620,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:11.142529+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:11.142529+0000","last_peered":"2024-09-15T21:19:11.142529+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:11.142529+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371177,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":8628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1016},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289423977,"num_objects":70661,"num_object_clones":0,"num_object_copies":141322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2502,"num_objects_unfound":0,"num_objects_dirty":70661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42156,"num_write_kb":168624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":3645440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582615040,"data_stored":582528065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70742,"ondisk_log_size":70742,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953496,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":390224,"kb_used_data":250240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93981616,"statfs":{"total":96636764160,"available":96237174784,"internally_reserved":0,"allocated":256245760,"data_stored":255990310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1,19],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149723,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410592,"kb_used_data":266512,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93961248,"statfs":{"total":96636764160,"available":96216317952,"internally_reserved":0,"allocated":272908288,"data_stored":272652838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":147517105},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626441,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159804,"kb_used_data":51756,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94212036,"statfs":{"total":96636764160,"available":96473124864,"internally_reserved":0,"allocated":52998144,"data_stored":52666156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":110620332},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272535552,"data_stored":272532073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":54206464,"data_stored":54126447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255873024,"data_stored":255869545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:18.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 85 20 72347 72327 3.3233 2.29688 0.00308101 0.0234633 2024-09-15T21:19:19.484 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 86 20 72703 72683 3.30082 1.39062 0.0106288 0.0236588 2024-09-15T21:19:20.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 87 20 73354 73334 3.29211 2.54297 0.0299633 0.0237213 2024-09-15T21:19:21.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:21.483 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 88 20 73858 73838 3.27707 1.96875 0.0591373 0.0238331 2024-09-15T21:19:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:21.881 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:21.907 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-15T21:19:20.332424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293700220,"num_objects":71707,"num_object_clones":0,"num_object_copies":143414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":71707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43202,"num_write_kb":172802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":964,"num_bytes_recovered":3944467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71789,"ondisk_log_size":71789,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":967536,"kb_used_data":575424,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282147984,"statfs":{"total":289910292480,"available":288919535616,"internally_reserved":0,"allocated":589234176,"data_stored":588391288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,11,8,13,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17723392,"num_objects":4327,"num_object_clones":0,"num_object_copies":8654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-388,"num_objects_unfound":0,"num_objects_dirty":4327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4327,"num_write_kb":17308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":1589248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013674"},"pg_stats":[{"pgid":"2.f","version":"44'4418","reported_seq":4489,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.795710+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:16.795710+0000","last_peered":"2024-09-15T21:19:16.795710+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:16.795710+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18096128,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4578","reported_seq":4669,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.362525+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:18.362525+0000","last_peered":"2024-09-15T21:19:18.362525+0000","last_clean":"2024-09-15T21:19:18.362525+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:18.362525+0000","last_undegraded":"2024-09-15T21:19:18.362525+0000","last_fullsized":"2024-09-15T21:19:18.362525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4578,"log_dups_size":0,"ondisk_log_size":4578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3304,"num_write_kb":13216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4506","reported_seq":6548,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:18.262701+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:18.262701+0000","last_peered":"2024-09-15T21:19:18.262701+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:18.262701+0000","last_undegraded":"2024-09-15T21:19:18.262701+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2429},{"shards":"2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4575","reported_seq":4639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.349480+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:18.349480+0000","last_peered":"2024-09-15T21:19:18.349480+0000","last_clean":"2024-09-15T21:19:18.349480+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:18.349480+0000","last_undegraded":"2024-09-15T21:19:18.349480+0000","last_fullsized":"2024-09-15T21:19:18.349480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4492","reported_seq":4605,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:16.757383+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:16.757383+0000","last_peered":"2024-09-15T21:19:16.757383+0000","last_clean":"2024-09-15T21:19:16.757383+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:16.757383+0000","last_undegraded":"2024-09-15T21:19:16.757383+0000","last_fullsized":"2024-09-15T21:19:16.757383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4492,"log_dups_size":0,"ondisk_log_size":4492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18370560,"num_objects":4485,"num_object_clones":0,"num_object_copies":8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2350,"num_write_kb":9400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4473","reported_seq":4597,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.514490+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:16.514490+0000","last_peered":"2024-09-15T21:19:16.514490+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:16.514490+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4473,"log_dups_size":0,"ondisk_log_size":4473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300928,"num_objects":4468,"num_object_clones":0,"num_object_copies":8936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1898,"num_write_kb":7592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4544","reported_seq":4662,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.759897+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:16.759897+0000","last_peered":"2024-09-15T21:19:16.759897+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:16.759897+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4544,"log_dups_size":0,"ondisk_log_size":4544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18587648,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2180,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1067},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4460","reported_seq":4574,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:16.759963+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:16.759963+0000","last_peered":"2024-09-15T21:19:16.759963+0000","last_clean":"2024-09-15T21:19:16.759963+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:16.759963+0000","last_undegraded":"2024-09-15T21:19:16.759963+0000","last_fullsized":"2024-09-15T21:19:16.759963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18259968,"num_objects":4458,"num_object_clones":0,"num_object_copies":8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2245,"num_write_kb":8980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4453","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.244849+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:18.244849+0000","last_peered":"2024-09-15T21:19:18.244849+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:18.244849+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3039,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4417","reported_seq":4534,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.761226+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:16.761226+0000","last_peered":"2024-09-15T21:19:16.761226+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:16.761226+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4417,"log_dups_size":0,"ondisk_log_size":4417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18059264,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1068},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4418","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.072228+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:18.072228+0000","last_peered":"2024-09-15T21:19:18.072228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:18.072228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18075648,"num_objects":4413,"num_object_clones":0,"num_object_copies":8826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2969,"num_write_kb":11876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4513","reported_seq":4622,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.236066+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:18.236066+0000","last_peered":"2024-09-15T21:19:18.236066+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:18.236066+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2912,"num_write_kb":11648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4513","reported_seq":4636,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.535199+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:16.535199+0000","last_peered":"2024-09-15T21:19:16.535199+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:16.535199+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18477056,"num_objects":4511,"num_object_clones":0,"num_object_copies":9022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1820,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1078},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4461","reported_seq":4583,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.778783+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:16.778783+0000","last_peered":"2024-09-15T21:19:16.778783+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:16.778783+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2046,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1074},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4431","reported_seq":4549,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.769147+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:16.769147+0000","last_peered":"2024-09-15T21:19:16.769147+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:16.769147+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18132992,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2078,"num_write_kb":8312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1062},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4534","reported_seq":4657,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:16.764970+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:16.764970+0000","last_peered":"2024-09-15T21:19:16.764970+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:16.764970+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4534,"log_dups_size":0,"ondisk_log_size":4534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18522729,"num_objects":4523,"num_object_clones":0,"num_object_copies":9046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":8776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293700201,"num_objects":71705,"num_object_clones":0,"num_object_copies":143410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":71705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43200,"num_write_kb":172800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":590311424,"data_stored":590224449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71786,"ondisk_log_size":71786,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393128,"kb_used_data":253144,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93978712,"statfs":{"total":96636764160,"available":96234201088,"internally_reserved":0,"allocated":259219456,"data_stored":258964006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149724,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412508,"kb_used_data":268428,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93959332,"statfs":{"total":96636764160,"available":96214355968,"internally_reserved":0,"allocated":274870272,"data_stored":274614822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":147517105},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161900,"kb_used_data":53852,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94209940,"statfs":{"total":96636764160,"available":96470978560,"internally_reserved":0,"allocated":55144448,"data_stored":54812460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":110620332},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":274681856,"data_stored":274678377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56274944,"data_stored":56194927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259354624,"data_stored":259351145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:21.908 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:22.484 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 89 20 74447 74427 3.2661 2.30078 0.0153873 0.0239128 2024-09-15T21:19:22.551 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:22.552 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:22.600 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-15T21:19:22.332845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300663420,"num_objects":73407,"num_object_clones":0,"num_object_copies":146814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":73407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44902,"num_write_kb":179602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":964,"num_bytes_recovered":3944467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73489,"ondisk_log_size":73489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":979564,"kb_used_data":591612,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282135956,"statfs":{"total":289910292480,"available":288907218944,"internally_reserved":0,"allocated":605810688,"data_stored":604967800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":397200394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,9,10,18,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20606976,"num_objects":5031,"num_object_clones":0,"num_object_copies":10062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":5031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5031,"num_write_kb":20124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013614"},"pg_stats":[{"pgid":"2.f","version":"44'4574","reported_seq":4645,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.828600+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:21.828600+0000","last_peered":"2024-09-15T21:19:21.828600+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:21.828600+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4574,"log_dups_size":0,"ondisk_log_size":4574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735104,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4578","reported_seq":4669,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.362525+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:18.362525+0000","last_peered":"2024-09-15T21:19:18.362525+0000","last_clean":"2024-09-15T21:19:18.362525+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:18.362525+0000","last_undegraded":"2024-09-15T21:19:18.362525+0000","last_fullsized":"2024-09-15T21:19:18.362525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4578,"log_dups_size":0,"ondisk_log_size":4578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3304,"num_write_kb":13216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4506","reported_seq":6548,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:18.262701+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:18.262701+0000","last_peered":"2024-09-15T21:19:18.262701+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:18.262701+0000","last_undegraded":"2024-09-15T21:19:18.262701+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2429},{"shards":"2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4575","reported_seq":4639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.349480+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:18.349480+0000","last_peered":"2024-09-15T21:19:18.349480+0000","last_clean":"2024-09-15T21:19:18.349480+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:18.349480+0000","last_undegraded":"2024-09-15T21:19:18.349480+0000","last_fullsized":"2024-09-15T21:19:18.349480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4699","reported_seq":4811,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.778176+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:21.778176+0000","last_peered":"2024-09-15T21:19:21.778176+0000","last_clean":"2024-09-15T21:19:21.778176+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:21.778176+0000","last_undegraded":"2024-09-15T21:19:21.778176+0000","last_fullsized":"2024-09-15T21:19:21.778176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4649","reported_seq":4773,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.779691+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:21.779691+0000","last_peered":"2024-09-15T21:19:21.779691+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:21.779691+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1273},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4706","reported_seq":4824,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.727292+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:21.727292+0000","last_peered":"2024-09-15T21:19:21.727292+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:21.727292+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1229},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4619","reported_seq":4733,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.788482+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:21.788482+0000","last_peered":"2024-09-15T21:19:21.788482+0000","last_clean":"2024-09-15T21:19:21.788482+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:21.788482+0000","last_undegraded":"2024-09-15T21:19:21.788482+0000","last_fullsized":"2024-09-15T21:19:21.788482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2404,"num_write_kb":9616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4453","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.244849+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:18.244849+0000","last_peered":"2024-09-15T21:19:18.244849+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:18.244849+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3039,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4587","reported_seq":4703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.811573+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:21.811573+0000","last_peered":"2024-09-15T21:19:21.811573+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:21.811573+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4418","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.072228+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:18.072228+0000","last_peered":"2024-09-15T21:19:18.072228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:18.072228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18075648,"num_objects":4413,"num_object_clones":0,"num_object_copies":8826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2969,"num_write_kb":11876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4513","reported_seq":4622,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.236066+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:18.236066+0000","last_peered":"2024-09-15T21:19:18.236066+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:18.236066+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2912,"num_write_kb":11648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4658","reported_seq":4781,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.737493+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:21.737493+0000","last_peered":"2024-09-15T21:19:21.737493+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:21.737493+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1965,"num_write_kb":7860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1223},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4625","reported_seq":4749,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.758031+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:21.758031+0000","last_peered":"2024-09-15T21:19:21.758031+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:21.758031+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2210,"num_write_kb":8840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4618","reported_seq":4736,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.789600+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:21.789600+0000","last_peered":"2024-09-15T21:19:21.789600+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:21.789600+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18898944,"num_objects":4614,"num_object_clones":0,"num_object_copies":9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2265,"num_write_kb":9060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4708","reported_seq":4831,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.813085+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:21.813085+0000","last_peered":"2024-09-15T21:19:21.813085+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:21.813085+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19235433,"num_objects":4697,"num_object_clones":0,"num_object_copies":9394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300663401,"num_objects":73405,"num_object_clones":0,"num_object_copies":146810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":73405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44900,"num_write_kb":179600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":607043584,"data_stored":606956609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73486,"ondisk_log_size":73486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393128,"kb_used_data":253144,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93978712,"statfs":{"total":96636764160,"available":96234201088,"internally_reserved":0,"allocated":259219456,"data_stored":258964006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149725,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412244,"kb_used_data":276484,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93959596,"statfs":{"total":96636764160,"available":96214626304,"internally_reserved":0,"allocated":283119616,"data_stored":282864166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":138997425},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174192,"kb_used_data":61984,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94197648,"statfs":{"total":96636764160,"available":96458391552,"internally_reserved":0,"allocated":63471616,"data_stored":63139628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":283029504,"data_stored":283026025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64659456,"data_stored":64579439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259354624,"data_stored":259351145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4574", 'reported_seq': 4645, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.828600+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:21.828600+0000', 'last_peered': '2024-09-15T21:19:21.828600+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:21.828600+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4574, 'log_dups_size': 0, 'ondisk_log_size': 4574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18735104, 'num_objects': 4574, 'num_object_clones': 0, 'num_object_copies': 9148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4574, 'num_write_kb': 18296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1207}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4506", 'reported_seq': 6548, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:18.262701+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:18.262701+0000', 'last_peered': '2024-09-15T21:19:18.262701+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:18.262701+0000', 'last_undegraded': '2024-09-15T21:19:18.262701+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4506, 'log_dups_size': 0, 'ondisk_log_size': 4506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18440192, 'num_objects': 4502, 'num_object_clones': 0, 'num_object_copies': 9004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2429, 'num_objects_unfound': 0, 'num_objects_dirty': 4502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3229, 'num_write_kb': 12916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 963, 'num_bytes_recovered': 3944448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2429}, {'shards': '2', 'objects': 2073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:22.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4649", 'reported_seq': 4773, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.779691+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:21.779691+0000', 'last_peered': '2024-09-15T21:19:21.779691+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:21.779691+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4649, 'log_dups_size': 0, 'ondisk_log_size': 4649, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19021824, 'num_objects': 4644, 'num_object_clones': 0, 'num_object_copies': 9288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2074, 'num_write_kb': 8296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1273}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4706", 'reported_seq': 4824, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.727292+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:21.727292+0000', 'last_peered': '2024-09-15T21:19:21.727292+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:21.727292+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 4706, 'log_dups_size': 0, 'ondisk_log_size': 4706, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19251200, 'num_objects': 4700, 'num_object_clones': 0, 'num_object_copies': 9400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2342, 'num_write_kb': 9368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1229}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4453", 'reported_seq': 4572, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:18.244849+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:18.244849+0000', 'last_peered': '2024-09-15T21:19:18.244849+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:18.244849+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4453, 'log_dups_size': 0, 'ondisk_log_size': 4453, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18210816, 'num_objects': 4446, 'num_object_clones': 0, 'num_object_copies': 8892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3039, 'num_write_kb': 12156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1108}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4587", 'reported_seq': 4703, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.811573+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:21.811573+0000', 'last_peered': '2024-09-15T21:19:21.811573+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:21.811573+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4587, 'log_dups_size': 0, 'ondisk_log_size': 4587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18755584, 'num_objects': 4579, 'num_object_clones': 0, 'num_object_copies': 9158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2113, 'num_write_kb': 8452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1238}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:22.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4418", 'reported_seq': 4522, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:18.072228+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:18.072228+0000', 'last_peered': '2024-09-15T21:19:18.072228+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:18.072228+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4418, 'log_dups_size': 0, 'ondisk_log_size': 4418, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18075648, 'num_objects': 4413, 'num_object_clones': 0, 'num_object_copies': 8826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2969, 'num_write_kb': 11876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4513", 'reported_seq': 4622, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:18.236066+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:18.236066+0000', 'last_peered': '2024-09-15T21:19:18.236066+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:18.236066+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4513, 'log_dups_size': 0, 'ondisk_log_size': 4513, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18460672, 'num_objects': 4507, 'num_object_clones': 0, 'num_object_copies': 9014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2912, 'num_write_kb': 11648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1157}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4658", 'reported_seq': 4781, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.737493+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:21.737493+0000', 'last_peered': '2024-09-15T21:19:21.737493+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:21.737493+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 4658, 'log_dups_size': 0, 'ondisk_log_size': 4658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19070976, 'num_objects': 4656, 'num_object_clones': 0, 'num_object_copies': 9312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1965, 'num_write_kb': 7860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1223}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4625", 'reported_seq': 4749, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.758031+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:21.758031+0000', 'last_peered': '2024-09-15T21:19:21.758031+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:21.758031+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4625, 'log_dups_size': 0, 'ondisk_log_size': 4625, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18911232, 'num_objects': 4617, 'num_object_clones': 0, 'num_object_copies': 9234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2210, 'num_write_kb': 8840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1238}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4618", 'reported_seq': 4736, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.789600+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:21.789600+0000', 'last_peered': '2024-09-15T21:19:21.789600+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:21.789600+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4618, 'log_dups_size': 0, 'ondisk_log_size': 4618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18898944, 'num_objects': 4614, 'num_object_clones': 0, 'num_object_copies': 9228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2265, 'num_write_kb': 9060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1249}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:22.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4708", 'reported_seq': 4831, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:21.813085+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:21.813085+0000', 'last_peered': '2024-09-15T21:19:21.813085+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:21.813085+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 4708, 'log_dups_size': 0, 'ondisk_log_size': 4708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19235433, 'num_objects': 4697, 'num_object_clones': 0, 'num_object_copies': 9394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2368, 'num_write_kb': 9472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1227}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:22.605 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:23.484 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 90 20 75043 75023 3.25567 2.32812 0.0111132 0.0239862 2024-09-15T21:19:23.664 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:23.664 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/146814 objects degraded (25.319%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":176,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73407,"data_bytes":300663420,"bytes_used":1003073536,"bytes_avail":288907218944,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":146814,"degraded_ratio":0.25319111256419685,"misplaced_objects":2429,"misplaced_total":146814,"misplaced_ratio":0.016544743689293935,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59324,"recovering_keys_per_sec":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"write_bytes_sec":1715301,"read_op_per_sec":0,"write_op_per_sec":418},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:23.689 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:23.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:24.240 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:24.241 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:24.278 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-15T21:19:22.332845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300663420,"num_objects":73407,"num_object_clones":0,"num_object_copies":146814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":73407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44902,"num_write_kb":179602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":964,"num_bytes_recovered":3944467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73489,"ondisk_log_size":73489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":979564,"kb_used_data":591612,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282135956,"statfs":{"total":289910292480,"available":288907218944,"internally_reserved":0,"allocated":605810688,"data_stored":604967800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":397200394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,9,10,18,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20606976,"num_objects":5031,"num_object_clones":0,"num_object_copies":10062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":5031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5031,"num_write_kb":20124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013614"},"pg_stats":[{"pgid":"2.f","version":"44'4574","reported_seq":4645,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.828600+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:21.828600+0000","last_peered":"2024-09-15T21:19:21.828600+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:21.828600+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4574,"log_dups_size":0,"ondisk_log_size":4574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735104,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4578","reported_seq":4669,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.362525+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:18.362525+0000","last_peered":"2024-09-15T21:19:18.362525+0000","last_clean":"2024-09-15T21:19:18.362525+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:18.362525+0000","last_undegraded":"2024-09-15T21:19:18.362525+0000","last_fullsized":"2024-09-15T21:19:18.362525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4578,"log_dups_size":0,"ondisk_log_size":4578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3304,"num_write_kb":13216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4506","reported_seq":6548,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:18.262701+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:18.262701+0000","last_peered":"2024-09-15T21:19:18.262701+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:18.262701+0000","last_undegraded":"2024-09-15T21:19:18.262701+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4506,"log_dups_size":0,"ondisk_log_size":4506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2429},{"shards":"2","objects":2073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4575","reported_seq":4639,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:18.349480+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:18.349480+0000","last_peered":"2024-09-15T21:19:18.349480+0000","last_clean":"2024-09-15T21:19:18.349480+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:18.349480+0000","last_undegraded":"2024-09-15T21:19:18.349480+0000","last_fullsized":"2024-09-15T21:19:18.349480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4699","reported_seq":4811,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.778176+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:21.778176+0000","last_peered":"2024-09-15T21:19:21.778176+0000","last_clean":"2024-09-15T21:19:21.778176+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:21.778176+0000","last_undegraded":"2024-09-15T21:19:21.778176+0000","last_fullsized":"2024-09-15T21:19:21.778176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4649","reported_seq":4773,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.779691+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:21.779691+0000","last_peered":"2024-09-15T21:19:21.779691+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:21.779691+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1273},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4706","reported_seq":4824,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.727292+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:21.727292+0000","last_peered":"2024-09-15T21:19:21.727292+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:21.727292+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1229},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4619","reported_seq":4733,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.788482+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:21.788482+0000","last_peered":"2024-09-15T21:19:21.788482+0000","last_clean":"2024-09-15T21:19:21.788482+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:21.788482+0000","last_undegraded":"2024-09-15T21:19:21.788482+0000","last_fullsized":"2024-09-15T21:19:21.788482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2404,"num_write_kb":9616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4453","reported_seq":4572,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.244849+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:18.244849+0000","last_peered":"2024-09-15T21:19:18.244849+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:18.244849+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18210816,"num_objects":4446,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3039,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4587","reported_seq":4703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.811573+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:21.811573+0000","last_peered":"2024-09-15T21:19:21.811573+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:21.811573+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4418","reported_seq":4522,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.072228+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:18.072228+0000","last_peered":"2024-09-15T21:19:18.072228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:18.072228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18075648,"num_objects":4413,"num_object_clones":0,"num_object_copies":8826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2969,"num_write_kb":11876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4513","reported_seq":4622,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:18.236066+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:18.236066+0000","last_peered":"2024-09-15T21:19:18.236066+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:18.236066+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2912,"num_write_kb":11648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1157}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4658","reported_seq":4781,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.737493+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:21.737493+0000","last_peered":"2024-09-15T21:19:21.737493+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:21.737493+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1965,"num_write_kb":7860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1223},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4625","reported_seq":4749,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.758031+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:21.758031+0000","last_peered":"2024-09-15T21:19:21.758031+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:21.758031+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2210,"num_write_kb":8840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4618","reported_seq":4736,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.789600+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:21.789600+0000","last_peered":"2024-09-15T21:19:21.789600+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:21.789600+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18898944,"num_objects":4614,"num_object_clones":0,"num_object_copies":9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2265,"num_write_kb":9060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4708","reported_seq":4831,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.813085+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:21.813085+0000","last_peered":"2024-09-15T21:19:21.813085+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:21.813085+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19235433,"num_objects":4697,"num_object_clones":0,"num_object_copies":9394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300663401,"num_objects":73405,"num_object_clones":0,"num_object_copies":146810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2429,"num_objects_unfound":0,"num_objects_dirty":73405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44900,"num_write_kb":179600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":3944448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":607043584,"data_stored":606956609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73486,"ondisk_log_size":73486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953497,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393128,"kb_used_data":253144,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93978712,"statfs":{"total":96636764160,"available":96234201088,"internally_reserved":0,"allocated":259219456,"data_stored":258964006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149725,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412244,"kb_used_data":276484,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93959596,"statfs":{"total":96636764160,"available":96214626304,"internally_reserved":0,"allocated":283119616,"data_stored":282864166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":138997425},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174192,"kb_used_data":61984,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94197648,"statfs":{"total":96636764160,"available":96458391552,"internally_reserved":0,"allocated":63471616,"data_stored":63139628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":283029504,"data_stored":283026025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64659456,"data_stored":64579439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259354624,"data_stored":259351145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:24.484 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 91 20 75452 75432 3.23745 1.59766 0.0162743 0.0241164 2024-09-15T21:19:25.484 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 92 20 75929 75909 3.22251 1.86328 0.00343227 0.0241633 2024-09-15T21:19:26.485 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 93 20 76229 76209 3.20043 1.17188 0.0545587 0.0244015 2024-09-15T21:19:27.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:27.485 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 94 20 76841 76821 3.19181 2.39062 0.0119945 0.0244651 2024-09-15T21:19:28.227 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:28.228 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:28.264 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-15T21:19:26.333667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304968316,"num_objects":74458,"num_object_clones":0,"num_object_copies":148916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":74458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45953,"num_write_kb":183806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1099,"num_bytes_recovered":4497427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74540,"ondisk_log_size":74540,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":989092,"kb_used_data":596980,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282126428,"statfs":{"total":289910292480,"available":288897462272,"internally_reserved":0,"allocated":611307520,"data_stored":610464632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,5,10,19,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15544320,"num_objects":3795,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-208,"num_objects_unfound":0,"num_objects_dirty":3795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3795,"num_write_kb":15180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013133"},"pg_stats":[{"pgid":"2.f","version":"44'4574","reported_seq":4645,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.828600+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:21.828600+0000","last_peered":"2024-09-15T21:19:21.828600+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:21.828600+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4574,"log_dups_size":0,"ondisk_log_size":4574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735104,"num_objects":4574,"num_object_clones":0,"num_object_copies":9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1207},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4745","reported_seq":4836,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:23.586388+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:23.586388+0000","last_peered":"2024-09-15T21:19:23.586388+0000","last_clean":"2024-09-15T21:19:23.586388+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:23.586388+0000","last_undegraded":"2024-09-15T21:19:23.586388+0000","last_fullsized":"2024-09-15T21:19:23.586388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4745,"log_dups_size":0,"ondisk_log_size":4745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19410944,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3471,"num_write_kb":13884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4696","reported_seq":7008,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:23.748807+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:23.748807+0000","last_peered":"2024-09-15T21:19:23.748807+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:23.748807+0000","last_undegraded":"2024-09-15T21:19:23.748807+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2294},{"shards":"2","objects":2398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4743","reported_seq":4807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:23.563120+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:23.563120+0000","last_peered":"2024-09-15T21:19:23.563120+0000","last_clean":"2024-09-15T21:19:23.563120+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:23.563120+0000","last_undegraded":"2024-09-15T21:19:23.563120+0000","last_fullsized":"2024-09-15T21:19:23.563120+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4743,"num_write_kb":18972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4699","reported_seq":4811,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.778176+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:21.778176+0000","last_peered":"2024-09-15T21:19:21.778176+0000","last_clean":"2024-09-15T21:19:21.778176+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:21.778176+0000","last_undegraded":"2024-09-15T21:19:21.778176+0000","last_fullsized":"2024-09-15T21:19:21.778176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4699,"log_dups_size":0,"ondisk_log_size":4699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4649","reported_seq":4773,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.779691+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:21.779691+0000","last_peered":"2024-09-15T21:19:21.779691+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:21.779691+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19021824,"num_objects":4644,"num_object_clones":0,"num_object_copies":9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2074,"num_write_kb":8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1273},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4706","reported_seq":4824,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.727292+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:21.727292+0000","last_peered":"2024-09-15T21:19:21.727292+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:21.727292+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19251200,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1229},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4619","reported_seq":4733,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:21.788482+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:21.788482+0000","last_peered":"2024-09-15T21:19:21.788482+0000","last_clean":"2024-09-15T21:19:21.788482+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:21.788482+0000","last_undegraded":"2024-09-15T21:19:21.788482+0000","last_fullsized":"2024-09-15T21:19:21.788482+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2404,"num_write_kb":9616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4630","reported_seq":4749,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.587388+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:23.587388+0000","last_peered":"2024-09-15T21:19:23.587388+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:23.587388+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4630,"log_dups_size":0,"ondisk_log_size":4630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18935808,"num_objects":4623,"num_object_clones":0,"num_object_copies":9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3216,"num_write_kb":12864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1285}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4587","reported_seq":4703,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.811573+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:21.811573+0000","last_peered":"2024-09-15T21:19:21.811573+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:21.811573+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4587,"log_dups_size":0,"ondisk_log_size":4587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2113,"num_write_kb":8452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":95,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.896193+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.896193+0000","last_peered":"2024-09-15T21:18:43.896193+0000","last_clean":"2024-09-15T21:18:43.896193+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.896193+0000","last_undegraded":"2024-09-15T21:18:43.896193+0000","last_fullsized":"2024-09-15T21:18:43.896193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4592","reported_seq":4696,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.618777+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:23.618777+0000","last_peered":"2024-09-15T21:19:23.618777+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:23.618777+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788352,"num_objects":4587,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3143,"num_write_kb":12572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4688","reported_seq":4797,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.607844+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:23.607844+0000","last_peered":"2024-09-15T21:19:23.607844+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:23.607844+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4688,"log_dups_size":0,"ondisk_log_size":4688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":12348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4658","reported_seq":4781,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.737493+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:21.737493+0000","last_peered":"2024-09-15T21:19:21.737493+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:21.737493+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1965,"num_write_kb":7860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1223},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4625","reported_seq":4749,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.758031+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:21.758031+0000","last_peered":"2024-09-15T21:19:21.758031+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:21.758031+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2210,"num_write_kb":8840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4618","reported_seq":4736,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.789600+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:21.789600+0000","last_peered":"2024-09-15T21:19:21.789600+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:21.789600+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18898944,"num_objects":4614,"num_object_clones":0,"num_object_copies":9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2265,"num_write_kb":9060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4708","reported_seq":4831,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:21.813085+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:21.813085+0000","last_peered":"2024-09-15T21:19:21.813085+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:21.813085+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19235433,"num_objects":4697,"num_object_clones":0,"num_object_copies":9394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304968297,"num_objects":74456,"num_object_clones":0,"num_object_copies":148912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":74456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45951,"num_write_kb":183804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612675584,"data_stored":612588609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74537,"ondisk_log_size":74537,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953498,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":402656,"kb_used_data":258512,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93969184,"statfs":{"total":96636764160,"available":96224444416,"internally_reserved":0,"allocated":264716288,"data_stored":264460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147582637},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149725,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412244,"kb_used_data":276484,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93959596,"statfs":{"total":96636764160,"available":96214626304,"internally_reserved":0,"allocated":283119616,"data_stored":282864166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":138997425},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174192,"kb_used_data":61984,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94197648,"statfs":{"total":96636764160,"available":96458391552,"internally_reserved":0,"allocated":63471616,"data_stored":63139628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":283029504,"data_stored":283026025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64659456,"data_stored":64579439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264986624,"data_stored":264983145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:28.265 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:28.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 95 20 77444 77424 3.18298 2.35547 0.0606986 0.0245352 2024-09-15T21:19:28.954 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:28.954 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:28.971 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-15T21:19:28.334081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311161468,"num_objects":75970,"num_object_clones":0,"num_object_copies":151940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":75970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47465,"num_write_kb":189854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1099,"num_bytes_recovered":4497427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76052,"ondisk_log_size":76052,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1000432,"kb_used_data":612480,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282115088,"statfs":{"total":289910292480,"available":288885850112,"internally_reserved":0,"allocated":627179520,"data_stored":626336632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":397200394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,4,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-208,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5307,"num_write_kb":21228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011468"},"pg_stats":[{"pgid":"2.f","version":"44'4708","reported_seq":4779,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.817584+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:26.817584+0000","last_peered":"2024-09-15T21:19:26.817584+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:26.817584+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19283968,"num_objects":4708,"num_object_clones":0,"num_object_copies":9416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4745","reported_seq":4836,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:23.586388+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:23.586388+0000","last_peered":"2024-09-15T21:19:23.586388+0000","last_clean":"2024-09-15T21:19:23.586388+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:23.586388+0000","last_undegraded":"2024-09-15T21:19:23.586388+0000","last_fullsized":"2024-09-15T21:19:23.586388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4745,"log_dups_size":0,"ondisk_log_size":4745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19410944,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3471,"num_write_kb":13884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4696","reported_seq":7008,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:23.748807+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:23.748807+0000","last_peered":"2024-09-15T21:19:23.748807+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:23.748807+0000","last_undegraded":"2024-09-15T21:19:23.748807+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218432,"num_objects":4692,"num_object_clones":0,"num_object_copies":9384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":4692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2294},{"shards":"2","objects":2398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4743","reported_seq":4807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:23.563120+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:23.563120+0000","last_peered":"2024-09-15T21:19:23.563120+0000","last_clean":"2024-09-15T21:19:23.563120+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:23.563120+0000","last_undegraded":"2024-09-15T21:19:23.563120+0000","last_fullsized":"2024-09-15T21:19:23.563120+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4743,"num_write_kb":18972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4858","reported_seq":4971,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:26.830375+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:26.830375+0000","last_peered":"2024-09-15T21:19:26.830375+0000","last_clean":"2024-09-15T21:19:26.830375+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:26.830375+0000","last_undegraded":"2024-09-15T21:19:26.830375+0000","last_fullsized":"2024-09-15T21:19:26.830375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4858,"log_dups_size":0,"ondisk_log_size":4858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2716,"num_write_kb":10864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4797","reported_seq":4921,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.831756+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:26.831756+0000","last_peered":"2024-09-15T21:19:26.831756+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:26.831756+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1421},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4880","reported_seq":4998,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.849065+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:26.849065+0000","last_peered":"2024-09-15T21:19:26.849065+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:26.849065+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4880,"log_dups_size":0,"ondisk_log_size":4880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19963904,"num_objects":4874,"num_object_clones":0,"num_object_copies":9748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4747","reported_seq":4861,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:26.773734+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:26.773734+0000","last_peered":"2024-09-15T21:19:26.773734+0000","last_clean":"2024-09-15T21:19:26.773734+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:26.773734+0000","last_undegraded":"2024-09-15T21:19:26.773734+0000","last_fullsized":"2024-09-15T21:19:26.773734+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2532,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4630","reported_seq":4749,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.587388+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:23.587388+0000","last_peered":"2024-09-15T21:19:23.587388+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:23.587388+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4630,"log_dups_size":0,"ondisk_log_size":4630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18935808,"num_objects":4623,"num_object_clones":0,"num_object_copies":9246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3216,"num_write_kb":12864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1285}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4748","reported_seq":4865,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.855156+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:26.855156+0000","last_peered":"2024-09-15T21:19:26.855156+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:26.855156+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19415040,"num_objects":4740,"num_object_clones":0,"num_object_copies":9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1399},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4592","reported_seq":4696,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.618777+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:23.618777+0000","last_peered":"2024-09-15T21:19:23.618777+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:23.618777+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788352,"num_objects":4587,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3143,"num_write_kb":12572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4688","reported_seq":4797,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:23.607844+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:23.607844+0000","last_peered":"2024-09-15T21:19:23.607844+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:23.607844+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4688,"log_dups_size":0,"ondisk_log_size":4688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":12348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4805","reported_seq":4928,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.814921+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:26.814921+0000","last_peered":"2024-09-15T21:19:26.814921+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:26.814921+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4805,"log_dups_size":0,"ondisk_log_size":4805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1370},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4778","reported_seq":4902,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.795582+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:26.795582+0000","last_peered":"2024-09-15T21:19:26.795582+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:26.795582+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4778,"log_dups_size":0,"ondisk_log_size":4778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2363,"num_write_kb":9452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1391},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4782","reported_seq":4897,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.817625+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:26.817625+0000","last_peered":"2024-09-15T21:19:26.817625+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:26.817625+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570688,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1413},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4852","reported_seq":4975,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.840534+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:26.840534+0000","last_peered":"2024-09-15T21:19:26.840534+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:26.840534+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19825257,"num_objects":4841,"num_object_clones":0,"num_object_copies":9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":10048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311161449,"num_objects":75968,"num_object_clones":0,"num_object_copies":151936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2294,"num_objects_unfound":0,"num_objects_dirty":75968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47463,"num_write_kb":189852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":628359168,"data_stored":628275672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76049,"ondisk_log_size":76049,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953498,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":402656,"kb_used_data":258512,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93969184,"statfs":{"total":96636764160,"available":96224444416,"internally_reserved":0,"allocated":264716288,"data_stored":264460838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147582637},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149726,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423556,"kb_used_data":283636,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93948284,"statfs":{"total":96636764160,"available":96203042816,"internally_reserved":0,"allocated":290443264,"data_stored":290187814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":143257265},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174220,"kb_used_data":70332,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94197620,"statfs":{"total":96636764160,"available":96458362880,"internally_reserved":0,"allocated":72019968,"data_stored":71687980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":106360492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":290402304,"data_stored":290398825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72970240,"data_stored":72893702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264986624,"data_stored":264983145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:28.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:28.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4708", 'reported_seq': 4779, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.817584+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:26.817584+0000', 'last_peered': '2024-09-15T21:19:26.817584+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:26.817584+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4708, 'log_dups_size': 0, 'ondisk_log_size': 4708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19283968, 'num_objects': 4708, 'num_object_clones': 0, 'num_object_copies': 9416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4708, 'num_write_kb': 18832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1341}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.972 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4696", 'reported_seq': 7008, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:23.748807+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:23.748807+0000', 'last_peered': '2024-09-15T21:19:23.748807+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:23.748807+0000', 'last_undegraded': '2024-09-15T21:19:23.748807+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4696, 'log_dups_size': 0, 'ondisk_log_size': 4696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19218432, 'num_objects': 4692, 'num_object_clones': 0, 'num_object_copies': 9384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2294, 'num_objects_unfound': 0, 'num_objects_dirty': 4692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3419, 'num_write_kb': 13676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1098, 'num_bytes_recovered': 4497408, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2294}, {'shards': '2', 'objects': 2398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4797", 'reported_seq': 4921, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.831756+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:26.831756+0000', 'last_peered': '2024-09-15T21:19:26.831756+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:26.831756+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4797, 'log_dups_size': 0, 'ondisk_log_size': 4797, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19628032, 'num_objects': 4792, 'num_object_clones': 0, 'num_object_copies': 9584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2222, 'num_write_kb': 8888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1421}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'4880", 'reported_seq': 4998, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.849065+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:26.849065+0000', 'last_peered': '2024-09-15T21:19:26.849065+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:26.849065+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 4880, 'log_dups_size': 0, 'ondisk_log_size': 4880, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19963904, 'num_objects': 4874, 'num_object_clones': 0, 'num_object_copies': 9748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2516, 'num_write_kb': 10064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1403}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4630", 'reported_seq': 4749, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:23.587388+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:23.587388+0000', 'last_peered': '2024-09-15T21:19:23.587388+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:23.587388+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4630, 'log_dups_size': 0, 'ondisk_log_size': 4630, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18935808, 'num_objects': 4623, 'num_object_clones': 0, 'num_object_copies': 9246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3216, 'num_write_kb': 12864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1285}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4748", 'reported_seq': 4865, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.855156+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:26.855156+0000', 'last_peered': '2024-09-15T21:19:26.855156+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:26.855156+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4748, 'log_dups_size': 0, 'ondisk_log_size': 4748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19415040, 'num_objects': 4740, 'num_object_clones': 0, 'num_object_copies': 9480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2274, 'num_write_kb': 9096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1399}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:28.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4592", 'reported_seq': 4696, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:23.618777+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:23.618777+0000', 'last_peered': '2024-09-15T21:19:23.618777+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:23.618777+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4592, 'log_dups_size': 0, 'ondisk_log_size': 4592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18788352, 'num_objects': 4587, 'num_object_clones': 0, 'num_object_copies': 9174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3143, 'num_write_kb': 12572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1321}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4688", 'reported_seq': 4797, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:23.607844+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:23.607844+0000', 'last_peered': '2024-09-15T21:19:23.607844+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:23.607844+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4688, 'log_dups_size': 0, 'ondisk_log_size': 4688, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19177472, 'num_objects': 4682, 'num_object_clones': 0, 'num_object_copies': 9364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3087, 'num_write_kb': 12348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1332}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'4805", 'reported_seq': 4928, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.814921+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:26.814921+0000', 'last_peered': '2024-09-15T21:19:26.814921+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:26.814921+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 4805, 'log_dups_size': 0, 'ondisk_log_size': 4805, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19673088, 'num_objects': 4803, 'num_object_clones': 0, 'num_object_copies': 9606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2112, 'num_write_kb': 8448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1370}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4778", 'reported_seq': 4902, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.795582+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:26.795582+0000', 'last_peered': '2024-09-15T21:19:26.795582+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:26.795582+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4778, 'log_dups_size': 0, 'ondisk_log_size': 4778, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19537920, 'num_objects': 4770, 'num_object_clones': 0, 'num_object_copies': 9540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2363, 'num_write_kb': 9452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1391}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4782", 'reported_seq': 4897, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.817625+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:26.817625+0000', 'last_peered': '2024-09-15T21:19:26.817625+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:26.817625+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4782, 'log_dups_size': 0, 'ondisk_log_size': 4782, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19570688, 'num_objects': 4778, 'num_object_clones': 0, 'num_object_copies': 9556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2429, 'num_write_kb': 9716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1413}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:28.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'4852", 'reported_seq': 4975, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:26.840534+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:26.840534+0000', 'last_peered': '2024-09-15T21:19:26.840534+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:26.840534+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 4852, 'log_dups_size': 0, 'ondisk_log_size': 4852, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19825257, 'num_objects': 4841, 'num_object_clones': 0, 'num_object_copies': 9682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2512, 'num_write_kb': 10048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1371}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:28.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:29.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 96 20 77927 77907 3.16948 1.88672 0.0604756 0.0246401 2024-09-15T21:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:29.836 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/151940 objects degraded (24.465%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75970,"data_bytes":311161468,"bytes_used":1024442368,"bytes_avail":288885850112,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":151940,"degraded_ratio":0.24464920363301304,"misplaced_objects":2294,"misplaced_total":151940,"misplaced_ratio":0.015098065025668028,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70929,"recovering_keys_per_sec":0,"num_objects_recovered":208,"num_bytes_recovered":851968,"num_keys_recovered":0,"write_bytes_sec":1809726,"read_op_per_sec":0,"write_op_per_sec":441},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:18:22.301191+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:29.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:29.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:30.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 97 20 78510 78490 3.16028 2.27734 0.0384611 0.0247116 2024-09-15T21:19:30.767 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:30.767 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:30.891 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-15T21:19:30.334549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":315028092,"num_objects":76914,"num_object_clones":0,"num_object_copies":153828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2085,"num_objects_unfound":0,"num_objects_dirty":76914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48409,"num_write_kb":193630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1309,"num_bytes_recovered":5353491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76996,"ondisk_log_size":76996,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1009276,"kb_used_data":617164,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282106244,"statfs":{"total":289910292480,"available":288876793856,"internally_reserved":0,"allocated":631975936,"data_stored":631133048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":401460234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,6,9,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23887872,"num_objects":5832,"num_object_clones":0,"num_object_copies":11664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-417,"num_objects_unfound":0,"num_objects_dirty":5832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":1708032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009592"},"pg_stats":[{"pgid":"2.f","version":"44'4708","reported_seq":4779,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.817584+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:26.817584+0000","last_peered":"2024-09-15T21:19:26.817584+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:26.817584+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19283968,"num_objects":4708,"num_object_clones":0,"num_object_copies":9416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'4909","reported_seq":5000,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.752179+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:28.752179+0000","last_peered":"2024-09-15T21:19:28.752179+0000","last_clean":"2024-09-15T21:19:28.752179+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:28.752179+0000","last_undegraded":"2024-09-15T21:19:28.752179+0000","last_fullsized":"2024-09-15T21:19:28.752179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3635,"num_write_kb":14540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4855","reported_seq":7587,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:28.745189+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:28.745189+0000","last_peered":"2024-09-15T21:19:28.745189+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:28.745189+0000","last_undegraded":"2024-09-15T21:19:28.745189+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2085,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3578,"num_write_kb":14312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1308,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2085},{"shards":"2","objects":2766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'4899","reported_seq":4963,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.744474+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:28.744474+0000","last_peered":"2024-09-15T21:19:28.744474+0000","last_clean":"2024-09-15T21:19:28.744474+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:28.744474+0000","last_undegraded":"2024-09-15T21:19:28.744474+0000","last_fullsized":"2024-09-15T21:19:28.744474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4899,"log_dups_size":0,"ondisk_log_size":4899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20066304,"num_objects":4899,"num_object_clones":0,"num_object_copies":9798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4899,"num_write_kb":19596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'4858","reported_seq":4971,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:26.830375+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:26.830375+0000","last_peered":"2024-09-15T21:19:26.830375+0000","last_clean":"2024-09-15T21:19:26.830375+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:26.830375+0000","last_undegraded":"2024-09-15T21:19:26.830375+0000","last_fullsized":"2024-09-15T21:19:26.830375+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":4858,"log_dups_size":0,"ondisk_log_size":4858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2716,"num_write_kb":10864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4797","reported_seq":4921,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.831756+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:26.831756+0000","last_peered":"2024-09-15T21:19:26.831756+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:26.831756+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1421},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'4880","reported_seq":4998,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.849065+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:26.849065+0000","last_peered":"2024-09-15T21:19:26.849065+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:26.849065+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":4880,"log_dups_size":0,"ondisk_log_size":4880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19963904,"num_objects":4874,"num_object_clones":0,"num_object_copies":9748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2516,"num_write_kb":10064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1403},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4747","reported_seq":4861,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:26.773734+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:26.773734+0000","last_peered":"2024-09-15T21:19:26.773734+0000","last_clean":"2024-09-15T21:19:26.773734+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:26.773734+0000","last_undegraded":"2024-09-15T21:19:26.773734+0000","last_fullsized":"2024-09-15T21:19:26.773734+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4747,"log_dups_size":0,"ondisk_log_size":4747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2532,"num_write_kb":10128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4782","reported_seq":4901,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:28.732788+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:28.732788+0000","last_peered":"2024-09-15T21:19:28.732788+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:28.732788+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19558400,"num_objects":4775,"num_object_clones":0,"num_object_copies":9550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3368,"num_write_kb":13472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4748","reported_seq":4865,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.855156+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:26.855156+0000","last_peered":"2024-09-15T21:19:26.855156+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:26.855156+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4748,"log_dups_size":0,"ondisk_log_size":4748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19415040,"num_objects":4740,"num_object_clones":0,"num_object_copies":9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2274,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1399},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4761","reported_seq":4859,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:28.739900+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:28.739900+0000","last_peered":"2024-09-15T21:19:28.739900+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:28.739900+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4761,"log_dups_size":0,"ondisk_log_size":4761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19480576,"num_objects":4756,"num_object_clones":0,"num_object_copies":9512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3312,"num_write_kb":13248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1490}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4832","reported_seq":4941,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:28.694158+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:28.694158+0000","last_peered":"2024-09-15T21:19:28.694158+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:28.694158+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767296,"num_objects":4826,"num_object_clones":0,"num_object_copies":9652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'4805","reported_seq":4928,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.814921+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:26.814921+0000","last_peered":"2024-09-15T21:19:26.814921+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:26.814921+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":4805,"log_dups_size":0,"ondisk_log_size":4805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19673088,"num_objects":4803,"num_object_clones":0,"num_object_copies":9606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1370},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4778","reported_seq":4902,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.795582+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:26.795582+0000","last_peered":"2024-09-15T21:19:26.795582+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:26.795582+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4778,"log_dups_size":0,"ondisk_log_size":4778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2363,"num_write_kb":9452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1391},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4782","reported_seq":4897,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.817625+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:26.817625+0000","last_peered":"2024-09-15T21:19:26.817625+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:26.817625+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570688,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1413},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'4852","reported_seq":4975,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:26.840534+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:26.840534+0000","last_peered":"2024-09-15T21:19:26.840534+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:26.840534+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19825257,"num_objects":4841,"num_object_clones":0,"num_object_copies":9682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":10048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":315028073,"num_objects":76912,"num_object_clones":0,"num_object_copies":153824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":2085,"num_objects_unfound":0,"num_objects_dirty":76912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48407,"num_write_kb":193628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1308,"num_bytes_recovered":5353472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":633438208,"data_stored":633354712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76993,"ondisk_log_size":76993,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953499,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411500,"kb_used_data":263196,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93960340,"statfs":{"total":96636764160,"available":96215388160,"internally_reserved":0,"allocated":269512704,"data_stored":269257254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149726,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423556,"kb_used_data":283636,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93948284,"statfs":{"total":96636764160,"available":96203042816,"internally_reserved":0,"allocated":290443264,"data_stored":290187814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":143257265},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174220,"kb_used_data":70332,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94197620,"statfs":{"total":96636764160,"available":96458362880,"internally_reserved":0,"allocated":72019968,"data_stored":71687980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":106360492},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":290402304,"data_stored":290398825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72970240,"data_stored":72893702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270065664,"data_stored":270062185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:31.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 98 20 79123 79103 3.15247 2.39453 0.0189019 0.0247736 2024-09-15T21:19:32.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 99 20 79505 79485 3.1357 1.49219 0.0155717 0.024859 2024-09-15T21:19:33.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:19:33.485221+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0249924 2024-09-15T21:19:33.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:19:33.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 100 20 79997 79977 3.12356 1.92188 0.027752 0.0249924 2024-09-15T21:19:33.892 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:34.486 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 101 20 80551 80531 3.11406 2.16406 0.0398497 0.0250767 2024-09-15T21:19:34.584 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:34.584 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:34.645 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-15T21:19:34.335335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325943932,"num_objects":79579,"num_object_clones":0,"num_object_copies":159158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51074,"num_write_kb":204290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1559,"num_bytes_recovered":6377491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79661,"ondisk_log_size":79661,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1037840,"kb_used_data":641568,"kb_used_omap":12,"kb_used_meta":396211,"kb_avail":282077680,"statfs":{"total":289910292480,"available":288847544320,"internally_reserved":0,"allocated":656965632,"data_stored":656126223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":405720074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,5,10,2,13,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-594,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6172,"num_write_kb":24688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":595,"num_bytes_recovered":2433024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002490"},"pg_stats":[{"pgid":"2.f","version":"44'4895","reported_seq":4964,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.847611+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:31.847611+0000","last_peered":"2024-09-15T21:19:31.847611+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:31.847611+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":19580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1528},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5064","reported_seq":5155,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760573+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:33.760573+0000","last_peered":"2024-09-15T21:19:33.760573+0000","last_clean":"2024-09-15T21:19:33.760573+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:33.760573+0000","last_undegraded":"2024-09-15T21:19:33.760573+0000","last_fullsized":"2024-09-15T21:19:33.760573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4998","reported_seq":8230,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:33.785299+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:33.785299+0000","last_peered":"2024-09-15T21:19:33.785299+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:33.785299+0000","last_undegraded":"2024-09-15T21:19:33.785299+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4998,"log_dups_size":0,"ondisk_log_size":4998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20455424,"num_objects":4994,"num_object_clones":0,"num_object_copies":9988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":4994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3721,"num_write_kb":14884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1835},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5071","reported_seq":5135,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760746+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:33.760746+0000","last_peered":"2024-09-15T21:19:33.760746+0000","last_clean":"2024-09-15T21:19:33.760746+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:33.760746+0000","last_undegraded":"2024-09-15T21:19:33.760746+0000","last_fullsized":"2024-09-15T21:19:33.760746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5071,"log_dups_size":0,"ondisk_log_size":5071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5071,"num_write_kb":20284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5015","reported_seq":5128,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.641241+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:31.641241+0000","last_peered":"2024-09-15T21:19:31.641241+0000","last_clean":"2024-09-15T21:19:31.641241+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:31.641241+0000","last_undegraded":"2024-09-15T21:19:31.641241+0000","last_fullsized":"2024-09-15T21:19:31.641241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2873,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4947","reported_seq":5066,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619670+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:31.619670+0000","last_peered":"2024-09-15T21:19:31.619670+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:31.619670+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5067","reported_seq":5185,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619347+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:31.619347+0000","last_peered":"2024-09-15T21:19:31.619347+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:31.619347+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20729856,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2703,"num_write_kb":10812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4917","reported_seq":5032,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.668329+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:31.668329+0000","last_peered":"2024-09-15T21:19:31.668329+0000","last_clean":"2024-09-15T21:19:31.668329+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:31.668329+0000","last_undegraded":"2024-09-15T21:19:31.668329+0000","last_fullsized":"2024-09-15T21:19:31.668329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2702,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4953","reported_seq":5072,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.776829+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:33.776829+0000","last_peered":"2024-09-15T21:19:33.776829+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:33.776829+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4903","reported_seq":5021,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.455157+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:31.455157+0000","last_peered":"2024-09-15T21:19:31.455157+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:31.455157+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4903,"log_dups_size":0,"ondisk_log_size":4903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1554},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4929","reported_seq":5032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.766061+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:33.766061+0000","last_peered":"2024-09-15T21:19:33.766061+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:33.766061+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4977","reported_seq":5086,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.688689+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:33.688689+0000","last_peered":"2024-09-15T21:19:33.688689+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:33.688689+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5004","reported_seq":5127,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.614130+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:31.614130+0000","last_peered":"2024-09-15T21:19:31.614130+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:31.614130+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4949","reported_seq":5073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.582141+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:31.582141+0000","last_peered":"2024-09-15T21:19:31.582141+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:31.582141+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4954","reported_seq":5071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.853236+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:31.853236+0000","last_peered":"2024-09-15T21:19:31.853236+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:31.853236+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1585},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5015","reported_seq":5138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.463104+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:31.463104+0000","last_peered":"2024-09-15T21:19:31.463104+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:31.463104+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20492905,"num_objects":5004,"num_object_clones":0,"num_object_copies":10008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2675,"num_write_kb":10700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325943913,"num_objects":79577,"num_object_clones":0,"num_object_copies":159154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51072,"num_write_kb":204288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":656678912,"data_stored":656595416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79658,"ondisk_log_size":79658,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404216,"kb_used_data":268392,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93967624,"statfs":{"total":96636764160,"available":96222846976,"internally_reserved":0,"allocated":274833408,"data_stored":274577958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":139062957},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440444,"kb_used_data":292204,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93931396,"statfs":{"total":96636764160,"available":96185749504,"internally_reserved":0,"allocated":299216896,"data_stored":298961446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,1,5,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":193180,"kb_used_data":80972,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94178660,"statfs":{"total":96636764160,"available":96438947840,"internally_reserved":0,"allocated":82915328,"data_stored":82586819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298868736,"data_stored":298865257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82579456,"data_stored":82502918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275230720,"data_stored":275227241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:34.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:35.323 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:35.361 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-15T21:19:34.335335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325943932,"num_objects":79579,"num_object_clones":0,"num_object_copies":159158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51074,"num_write_kb":204290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1559,"num_bytes_recovered":6377491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79661,"ondisk_log_size":79661,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1037840,"kb_used_data":641568,"kb_used_omap":12,"kb_used_meta":396211,"kb_avail":282077680,"statfs":{"total":289910292480,"available":288847544320,"internally_reserved":0,"allocated":656965632,"data_stored":656126223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":405720074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,5,10,2,13,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25280512,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-594,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6172,"num_write_kb":24688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":595,"num_bytes_recovered":2433024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002490"},"pg_stats":[{"pgid":"2.f","version":"44'4895","reported_seq":4964,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.847611+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:31.847611+0000","last_peered":"2024-09-15T21:19:31.847611+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:31.847611+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":19580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1528},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5064","reported_seq":5155,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760573+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:33.760573+0000","last_peered":"2024-09-15T21:19:33.760573+0000","last_clean":"2024-09-15T21:19:33.760573+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:33.760573+0000","last_undegraded":"2024-09-15T21:19:33.760573+0000","last_fullsized":"2024-09-15T21:19:33.760573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4998","reported_seq":8230,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:33.785299+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:33.785299+0000","last_peered":"2024-09-15T21:19:33.785299+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:33.785299+0000","last_undegraded":"2024-09-15T21:19:33.785299+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4998,"log_dups_size":0,"ondisk_log_size":4998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20455424,"num_objects":4994,"num_object_clones":0,"num_object_copies":9988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":4994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3721,"num_write_kb":14884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1835},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5071","reported_seq":5135,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760746+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:33.760746+0000","last_peered":"2024-09-15T21:19:33.760746+0000","last_clean":"2024-09-15T21:19:33.760746+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:33.760746+0000","last_undegraded":"2024-09-15T21:19:33.760746+0000","last_fullsized":"2024-09-15T21:19:33.760746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5071,"log_dups_size":0,"ondisk_log_size":5071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5071,"num_write_kb":20284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5015","reported_seq":5128,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.641241+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:31.641241+0000","last_peered":"2024-09-15T21:19:31.641241+0000","last_clean":"2024-09-15T21:19:31.641241+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:31.641241+0000","last_undegraded":"2024-09-15T21:19:31.641241+0000","last_fullsized":"2024-09-15T21:19:31.641241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2873,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4947","reported_seq":5066,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619670+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:31.619670+0000","last_peered":"2024-09-15T21:19:31.619670+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:31.619670+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5067","reported_seq":5185,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619347+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:31.619347+0000","last_peered":"2024-09-15T21:19:31.619347+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:31.619347+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20729856,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2703,"num_write_kb":10812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4917","reported_seq":5032,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.668329+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:31.668329+0000","last_peered":"2024-09-15T21:19:31.668329+0000","last_clean":"2024-09-15T21:19:31.668329+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:31.668329+0000","last_undegraded":"2024-09-15T21:19:31.668329+0000","last_fullsized":"2024-09-15T21:19:31.668329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2702,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4953","reported_seq":5072,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.776829+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:33.776829+0000","last_peered":"2024-09-15T21:19:33.776829+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:33.776829+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4903","reported_seq":5021,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.455157+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:31.455157+0000","last_peered":"2024-09-15T21:19:31.455157+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:31.455157+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4903,"log_dups_size":0,"ondisk_log_size":4903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1554},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4929","reported_seq":5032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.766061+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:33.766061+0000","last_peered":"2024-09-15T21:19:33.766061+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:33.766061+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4977","reported_seq":5086,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.688689+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:33.688689+0000","last_peered":"2024-09-15T21:19:33.688689+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:33.688689+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5004","reported_seq":5127,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.614130+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:31.614130+0000","last_peered":"2024-09-15T21:19:31.614130+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:31.614130+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4949","reported_seq":5073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.582141+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:31.582141+0000","last_peered":"2024-09-15T21:19:31.582141+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:31.582141+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4954","reported_seq":5071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.853236+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:31.853236+0000","last_peered":"2024-09-15T21:19:31.853236+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:31.853236+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1585},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5015","reported_seq":5138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.463104+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:31.463104+0000","last_peered":"2024-09-15T21:19:31.463104+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:31.463104+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20492905,"num_objects":5004,"num_object_clones":0,"num_object_copies":10008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2675,"num_write_kb":10700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325943913,"num_objects":79577,"num_object_clones":0,"num_object_copies":159154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51072,"num_write_kb":204288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":656678912,"data_stored":656595416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79658,"ondisk_log_size":79658,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404216,"kb_used_data":268392,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93967624,"statfs":{"total":96636764160,"available":96222846976,"internally_reserved":0,"allocated":274833408,"data_stored":274577958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":139062957},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440444,"kb_used_data":292204,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93931396,"statfs":{"total":96636764160,"available":96185749504,"internally_reserved":0,"allocated":299216896,"data_stored":298961446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,1,5,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":193180,"kb_used_data":80972,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94178660,"statfs":{"total":96636764160,"available":96438947840,"internally_reserved":0,"allocated":82915328,"data_stored":82586819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298868736,"data_stored":298865257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82579456,"data_stored":82502918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275230720,"data_stored":275227241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:35.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:35.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'4895", 'reported_seq': 4964, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.847611+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:31.847611+0000', 'last_peered': '2024-09-15T21:19:31.847611+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:31.847611+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 4895, 'log_dups_size': 0, 'ondisk_log_size': 4895, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20049920, 'num_objects': 4895, 'num_object_clones': 0, 'num_object_copies': 9790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4895, 'num_write_kb': 19580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1528}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:35.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'4998", 'reported_seq': 8230, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:33.785299+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:33.785299+0000', 'last_peered': '2024-09-15T21:19:33.785299+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:33.785299+0000', 'last_undegraded': '2024-09-15T21:19:33.785299+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 4998, 'log_dups_size': 0, 'ondisk_log_size': 4998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20455424, 'num_objects': 4994, 'num_object_clones': 0, 'num_object_copies': 9988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1835, 'num_objects_unfound': 0, 'num_objects_dirty': 4994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3721, 'num_write_kb': 14884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1558, 'num_bytes_recovered': 6377472, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1835}, {'shards': '2', 'objects': 3159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'4947", 'reported_seq': 5066, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.619670+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:31.619670+0000', 'last_peered': '2024-09-15T21:19:31.619670+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:31.619670+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 4947, 'log_dups_size': 0, 'ondisk_log_size': 4947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20242432, 'num_objects': 4942, 'num_object_clones': 0, 'num_object_copies': 9884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2372, 'num_write_kb': 9488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1571}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5067", 'reported_seq': 5185, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.619347+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:31.619347+0000', 'last_peered': '2024-09-15T21:19:31.619347+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:31.619347+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 5067, 'log_dups_size': 0, 'ondisk_log_size': 5067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20729856, 'num_objects': 5061, 'num_object_clones': 0, 'num_object_copies': 10122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2703, 'num_write_kb': 10812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1590}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'4953", 'reported_seq': 5072, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:33.776829+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:33.776829+0000', 'last_peered': '2024-09-15T21:19:33.776829+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:33.776829+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 4953, 'log_dups_size': 0, 'ondisk_log_size': 4953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20258816, 'num_objects': 4946, 'num_object_clones': 0, 'num_object_copies': 9892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3539, 'num_write_kb': 14156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'4903", 'reported_seq': 5021, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.455157+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:31.455157+0000', 'last_peered': '2024-09-15T21:19:31.455157+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:31.455157+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 4903, 'log_dups_size': 0, 'ondisk_log_size': 4903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20049920, 'num_objects': 4895, 'num_object_clones': 0, 'num_object_copies': 9790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2429, 'num_write_kb': 9716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1554}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'4929", 'reported_seq': 5032, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:33.766061+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:33.766061+0000', 'last_peered': '2024-09-15T21:19:33.766061+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:33.766061+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 4929, 'log_dups_size': 0, 'ondisk_log_size': 4929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20168704, 'num_objects': 4924, 'num_object_clones': 0, 'num_object_copies': 9848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3480, 'num_write_kb': 13920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1658}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'4977", 'reported_seq': 5086, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:33.688689+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:33.688689+0000', 'last_peered': '2024-09-15T21:19:33.688689+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:33.688689+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 4977, 'log_dups_size': 0, 'ondisk_log_size': 4977, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20361216, 'num_objects': 4971, 'num_object_clones': 0, 'num_object_copies': 9942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3376, 'num_write_kb': 13504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1621}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5004", 'reported_seq': 5127, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.614130+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:31.614130+0000', 'last_peered': '2024-09-15T21:19:31.614130+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:31.614130+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 5004, 'log_dups_size': 0, 'ondisk_log_size': 5004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20488192, 'num_objects': 5002, 'num_object_clones': 0, 'num_object_copies': 10004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2311, 'num_write_kb': 9244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1569}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'4949", 'reported_seq': 5073, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.582141+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:31.582141+0000', 'last_peered': '2024-09-15T21:19:31.582141+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:31.582141+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 4949, 'log_dups_size': 0, 'ondisk_log_size': 4949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20238336, 'num_objects': 4941, 'num_object_clones': 0, 'num_object_copies': 9882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2534, 'num_write_kb': 10136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1562}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'4954", 'reported_seq': 5071, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.853236+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:31.853236+0000', 'last_peered': '2024-09-15T21:19:31.853236+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:31.853236+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 4954, 'log_dups_size': 0, 'ondisk_log_size': 4954, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20275200, 'num_objects': 4950, 'num_object_clones': 0, 'num_object_copies': 9900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2601, 'num_write_kb': 10404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1585}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:35.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5015", 'reported_seq': 5138, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:31.463104+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:31.463104+0000', 'last_peered': '2024-09-15T21:19:31.463104+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:31.463104+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 5015, 'log_dups_size': 0, 'ondisk_log_size': 5015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20492905, 'num_objects': 5004, 'num_object_clones': 0, 'num_object_copies': 10008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2675, 'num_write_kb': 10700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1534}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:35.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:35.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 102 20 81139 81119 3.10604 2.29688 0.0416853 0.0251436 2024-09-15T21:19:36.063 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:36.063 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/159158 objects degraded (23.355%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":189,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79579,"data_bytes":325943932,"bytes_used":1062748160,"bytes_avail":288847544320,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":159158,"degraded_ratio":0.23355407833724978,"misplaced_objects":1835,"misplaced_total":159158,"misplaced_ratio":0.011529423591651065,"recovering_objects_per_sec":49,"recovering_bytes_per_sec":202709,"recovering_keys_per_sec":0,"num_objects_recovered":595,"num_bytes_recovered":2433024,"num_keys_recovered":0,"write_bytes_sec":2106272,"read_op_per_sec":0,"write_op_per_sec":514},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:19:34.335412+0000","services":{"mon":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:36.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:36.131 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:36.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 103 20 81645 81625 3.09507 1.97656 0.0422482 0.0252318 2024-09-15T21:19:36.793 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:36.793 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:36.811 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-15T21:19:36.335760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325943932,"num_objects":79579,"num_object_clones":0,"num_object_copies":159158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51074,"num_write_kb":204290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1559,"num_bytes_recovered":6377491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79661,"ondisk_log_size":79661,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1037840,"kb_used_data":641568,"kb_used_omap":12,"kb_used_meta":396211,"kb_avail":282077680,"statfs":{"total":289910292480,"available":288847544320,"internally_reserved":0,"allocated":656965632,"data_stored":656126223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":405720074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,5,10,2,13,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-459,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5121,"num_write_kb":20484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":1880064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002458"},"pg_stats":[{"pgid":"2.f","version":"44'4895","reported_seq":4964,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.847611+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:31.847611+0000","last_peered":"2024-09-15T21:19:31.847611+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:31.847611+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":19580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1528},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5064","reported_seq":5155,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760573+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:33.760573+0000","last_peered":"2024-09-15T21:19:33.760573+0000","last_clean":"2024-09-15T21:19:33.760573+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:33.760573+0000","last_undegraded":"2024-09-15T21:19:33.760573+0000","last_fullsized":"2024-09-15T21:19:33.760573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3790,"num_write_kb":15160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'4998","reported_seq":8230,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:33.785299+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:33.785299+0000","last_peered":"2024-09-15T21:19:33.785299+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:33.785299+0000","last_undegraded":"2024-09-15T21:19:33.785299+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":4998,"log_dups_size":0,"ondisk_log_size":4998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20455424,"num_objects":4994,"num_object_clones":0,"num_object_copies":9988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":4994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3721,"num_write_kb":14884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1835},{"shards":"2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5071","reported_seq":5135,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:33.760746+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:33.760746+0000","last_peered":"2024-09-15T21:19:33.760746+0000","last_clean":"2024-09-15T21:19:33.760746+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:33.760746+0000","last_undegraded":"2024-09-15T21:19:33.760746+0000","last_fullsized":"2024-09-15T21:19:33.760746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5071,"log_dups_size":0,"ondisk_log_size":5071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5071,"num_write_kb":20284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5015","reported_seq":5128,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.641241+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:31.641241+0000","last_peered":"2024-09-15T21:19:31.641241+0000","last_clean":"2024-09-15T21:19:31.641241+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:31.641241+0000","last_undegraded":"2024-09-15T21:19:31.641241+0000","last_fullsized":"2024-09-15T21:19:31.641241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2873,"num_write_kb":11492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'4947","reported_seq":5066,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619670+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:31.619670+0000","last_peered":"2024-09-15T21:19:31.619670+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:31.619670+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":4947,"log_dups_size":0,"ondisk_log_size":4947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1571},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5067","reported_seq":5185,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.619347+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:31.619347+0000","last_peered":"2024-09-15T21:19:31.619347+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:31.619347+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20729856,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2703,"num_write_kb":10812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1590},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'4917","reported_seq":5032,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:31.668329+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:31.668329+0000","last_peered":"2024-09-15T21:19:31.668329+0000","last_clean":"2024-09-15T21:19:31.668329+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:31.668329+0000","last_undegraded":"2024-09-15T21:19:31.668329+0000","last_fullsized":"2024-09-15T21:19:31.668329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2702,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'4953","reported_seq":5072,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.776829+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:33.776829+0000","last_peered":"2024-09-15T21:19:33.776829+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:33.776829+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3539,"num_write_kb":14156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'4903","reported_seq":5021,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.455157+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:31.455157+0000","last_peered":"2024-09-15T21:19:31.455157+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:31.455157+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":4903,"log_dups_size":0,"ondisk_log_size":4903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2429,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1554},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'4929","reported_seq":5032,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.766061+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:33.766061+0000","last_peered":"2024-09-15T21:19:33.766061+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:33.766061+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'4977","reported_seq":5086,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:33.688689+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:33.688689+0000","last_peered":"2024-09-15T21:19:33.688689+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:33.688689+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20361216,"num_objects":4971,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5004","reported_seq":5127,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.614130+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:31.614130+0000","last_peered":"2024-09-15T21:19:31.614130+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:31.614130+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2311,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'4949","reported_seq":5073,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.582141+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:31.582141+0000","last_peered":"2024-09-15T21:19:31.582141+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:31.582141+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20238336,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'4954","reported_seq":5071,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.853236+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:31.853236+0000","last_peered":"2024-09-15T21:19:31.853236+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:31.853236+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1585},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5015","reported_seq":5138,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:31.463104+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:31.463104+0000","last_peered":"2024-09-15T21:19:31.463104+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:31.463104+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20492905,"num_objects":5004,"num_object_clones":0,"num_object_copies":10008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2675,"num_write_kb":10700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1534},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325943913,"num_objects":79577,"num_object_clones":0,"num_object_copies":159154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1835,"num_objects_unfound":0,"num_objects_dirty":79577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51072,"num_write_kb":204288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1558,"num_bytes_recovered":6377472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":656678912,"data_stored":656595416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79658,"ondisk_log_size":79658,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404216,"kb_used_data":268392,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93967624,"statfs":{"total":96636764160,"available":96222846976,"internally_reserved":0,"allocated":274833408,"data_stored":274577958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":139062957},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149727,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440444,"kb_used_data":292204,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93931396,"statfs":{"total":96636764160,"available":96185749504,"internally_reserved":0,"allocated":299216896,"data_stored":298961446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":151776945},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,1,5,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626445,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":193180,"kb_used_data":80972,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94178660,"statfs":{"total":96636764160,"available":96438947840,"internally_reserved":0,"allocated":82915328,"data_stored":82586819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":114880172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,4,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298868736,"data_stored":298865257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82579456,"data_stored":82502918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275230720,"data_stored":275227241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:37.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 104 20 82104 82084 3.08255 1.79297 0.0218718 0.0253153 2024-09-15T21:19:38.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 105 20 82541 82521 3.06945 1.70703 0.0353487 0.0254423 2024-09-15T21:19:39.487 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 106 20 83151 83131 3.06297 2.38281 0.0116969 0.0254979 2024-09-15T21:19:39.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:40.488 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 107 20 83706 83686 3.05459 2.16797 0.0127686 0.025565 2024-09-15T21:19:40.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:40.630 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:40.684 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-15T21:19:40.336847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336405116,"num_objects":82133,"num_object_clones":0,"num_object_copies":164266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":82133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53628,"num_write_kb":214506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1747,"num_bytes_recovered":7147539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82215,"ondisk_log_size":82215,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1058972,"kb_used_data":661292,"kb_used_omap":12,"kb_used_meta":397619,"kb_avail":282056548,"statfs":{"total":289910292480,"available":288825905152,"internally_reserved":0,"allocated":677163008,"data_stored":676323599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":407161866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,5,8,12,12,17],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-647,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":2650112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"2.f","version":"44'5058","reported_seq":5130,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.763588+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:36.763588+0000","last_peered":"2024-09-15T21:19:36.763588+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:36.763588+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5058,"log_dups_size":0,"ondisk_log_size":5058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5058,"num_write_kb":20232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1691},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5217","reported_seq":5308,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.773896+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:38.773896+0000","last_peered":"2024-09-15T21:19:38.773896+0000","last_clean":"2024-09-15T21:19:38.773896+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:38.773896+0000","last_undegraded":"2024-09-15T21:19:38.773896+0000","last_fullsized":"2024-09-15T21:19:38.773896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21344256,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":15772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5202","reported_seq":8810,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:38.759510+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:38.759510+0000","last_peered":"2024-09-15T21:19:38.759510+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:38.759510+0000","last_undegraded":"2024-09-15T21:19:38.759510+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5202,"log_dups_size":0,"ondisk_log_size":5202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3925,"num_write_kb":15700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":3551}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5235","reported_seq":5299,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.815216+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:38.815216+0000","last_peered":"2024-09-15T21:19:38.815216+0000","last_clean":"2024-09-15T21:19:38.815216+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:38.815216+0000","last_undegraded":"2024-09-15T21:19:38.815216+0000","last_fullsized":"2024-09-15T21:19:38.815216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21442560,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5179","reported_seq":5293,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:36.754754+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:36.754754+0000","last_peered":"2024-09-15T21:19:36.754754+0000","last_clean":"2024-09-15T21:19:36.754754+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:36.754754+0000","last_undegraded":"2024-09-15T21:19:36.754754+0000","last_fullsized":"2024-09-15T21:19:36.754754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3037,"num_write_kb":12148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5116","reported_seq":5240,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.743611+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:36.743611+0000","last_peered":"2024-09-15T21:19:36.743611+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:36.743611+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5116,"log_dups_size":0,"ondisk_log_size":5116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20934656,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":10164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1740},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5240","reported_seq":5358,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.721595+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:36.721595+0000","last_peered":"2024-09-15T21:19:36.721595+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:36.721595+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2876,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1763},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5065","reported_seq":5180,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:36.771580+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:36.771580+0000","last_peered":"2024-09-15T21:19:36.771580+0000","last_clean":"2024-09-15T21:19:36.771580+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:36.771580+0000","last_undegraded":"2024-09-15T21:19:36.771580+0000","last_fullsized":"2024-09-15T21:19:36.771580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738048,"num_objects":5063,"num_object_clones":0,"num_object_copies":10126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5091","reported_seq":5210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.743768+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:38.743768+0000","last_peered":"2024-09-15T21:19:38.743768+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:38.743768+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5064","reported_seq":5182,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.732697+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:36.732697+0000","last_peered":"2024-09-15T21:19:36.732697+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:36.732697+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":5056,"num_object_clones":0,"num_object_copies":10112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1715},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5077","reported_seq":5180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.817878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:38.817878+0000","last_peered":"2024-09-15T21:19:38.817878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:38.817878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3628,"num_write_kb":14512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1806}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5131","reported_seq":5241,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.834601+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:38.834601+0000","last_peered":"2024-09-15T21:19:38.834601+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:38.834601+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3530,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5170","reported_seq":5293,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.735079+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:36.735079+0000","last_peered":"2024-09-15T21:19:36.735079+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:36.735079+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5170,"log_dups_size":0,"ondisk_log_size":5170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":5168,"num_object_clones":0,"num_object_copies":10336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2477,"num_write_kb":9908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1735},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5092","reported_seq":5216,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.744821+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:36.744821+0000","last_peered":"2024-09-15T21:19:36.744821+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:36.744821+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5092,"log_dups_size":0,"ondisk_log_size":5092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2677,"num_write_kb":10708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5117","reported_seq":5236,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.768172+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:36.768172+0000","last_peered":"2024-09-15T21:19:36.768172+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:36.768172+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2764,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1748},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5158","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.785538+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:36.785538+0000","last_peered":"2024-09-15T21:19:36.785538+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:36.785538+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078633,"num_objects":5147,"num_object_clones":0,"num_object_copies":10294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2818,"num_write_kb":11272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336405097,"num_objects":82131,"num_object_clones":0,"num_object_copies":164262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":82131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53626,"num_write_kb":214504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678502400,"data_stored":678418904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82212,"ondisk_log_size":82212,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417812,"kb_used_data":273668,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93954028,"statfs":{"total":96636764160,"available":96208924672,"internally_reserved":0,"allocated":280236032,"data_stored":279980582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147582637},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149728,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453556,"kb_used_data":299748,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93918284,"statfs":{"total":96636764160,"available":96172322816,"internally_reserved":0,"allocated":306941952,"data_stored":306686502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":157478577},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,4,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":187604,"kb_used_data":87876,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94184236,"statfs":{"total":96636764160,"available":96444657664,"internally_reserved":0,"allocated":89985024,"data_stored":89656515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":102100652},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306724864,"data_stored":306721385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91303936,"data_stored":91227398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280473600,"data_stored":280470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:40.686 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:41.488 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 108 20 84274 84254 3.04685 2.21875 0.0273524 0.0256313 2024-09-15T21:19:41.550 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:41.550 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:41.639 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-15T21:19:40.336847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336405116,"num_objects":82133,"num_object_clones":0,"num_object_copies":164266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":82133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53628,"num_write_kb":214506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1747,"num_bytes_recovered":7147539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82215,"ondisk_log_size":82215,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1058972,"kb_used_data":661292,"kb_used_omap":12,"kb_used_meta":397619,"kb_avail":282056548,"statfs":{"total":289910292480,"available":288825905152,"internally_reserved":0,"allocated":677163008,"data_stored":676323599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":407161866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,5,8,12,12,17],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-647,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":2650112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"2.f","version":"44'5058","reported_seq":5130,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.763588+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:36.763588+0000","last_peered":"2024-09-15T21:19:36.763588+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:36.763588+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5058,"log_dups_size":0,"ondisk_log_size":5058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20717568,"num_objects":5058,"num_object_clones":0,"num_object_copies":10116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5058,"num_write_kb":20232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1691},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5217","reported_seq":5308,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.773896+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:38.773896+0000","last_peered":"2024-09-15T21:19:38.773896+0000","last_clean":"2024-09-15T21:19:38.773896+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:38.773896+0000","last_undegraded":"2024-09-15T21:19:38.773896+0000","last_fullsized":"2024-09-15T21:19:38.773896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21344256,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":15772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5202","reported_seq":8810,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:38.759510+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:38.759510+0000","last_peered":"2024-09-15T21:19:38.759510+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:38.759510+0000","last_undegraded":"2024-09-15T21:19:38.759510+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5202,"log_dups_size":0,"ondisk_log_size":5202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3925,"num_write_kb":15700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":3551}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5235","reported_seq":5299,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.815216+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:38.815216+0000","last_peered":"2024-09-15T21:19:38.815216+0000","last_clean":"2024-09-15T21:19:38.815216+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:38.815216+0000","last_undegraded":"2024-09-15T21:19:38.815216+0000","last_fullsized":"2024-09-15T21:19:38.815216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21442560,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5179","reported_seq":5293,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:36.754754+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:36.754754+0000","last_peered":"2024-09-15T21:19:36.754754+0000","last_clean":"2024-09-15T21:19:36.754754+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:36.754754+0000","last_undegraded":"2024-09-15T21:19:36.754754+0000","last_fullsized":"2024-09-15T21:19:36.754754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21184512,"num_objects":5172,"num_object_clones":0,"num_object_copies":10344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3037,"num_write_kb":12148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5116","reported_seq":5240,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.743611+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:36.743611+0000","last_peered":"2024-09-15T21:19:36.743611+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:36.743611+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5116,"log_dups_size":0,"ondisk_log_size":5116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20934656,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":10164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1740},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5240","reported_seq":5358,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.721595+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:36.721595+0000","last_peered":"2024-09-15T21:19:36.721595+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:36.721595+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21438464,"num_objects":5234,"num_object_clones":0,"num_object_copies":10468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2876,"num_write_kb":11504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1763},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5065","reported_seq":5180,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:36.771580+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:36.771580+0000","last_peered":"2024-09-15T21:19:36.771580+0000","last_clean":"2024-09-15T21:19:36.771580+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:36.771580+0000","last_undegraded":"2024-09-15T21:19:36.771580+0000","last_fullsized":"2024-09-15T21:19:36.771580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5065,"log_dups_size":0,"ondisk_log_size":5065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20738048,"num_objects":5063,"num_object_clones":0,"num_object_copies":10126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5091","reported_seq":5210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.743768+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:38.743768+0000","last_peered":"2024-09-15T21:19:38.743768+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:38.743768+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5064","reported_seq":5182,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.732697+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:36.732697+0000","last_peered":"2024-09-15T21:19:36.732697+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:36.732697+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5064,"log_dups_size":0,"ondisk_log_size":5064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":5056,"num_object_clones":0,"num_object_copies":10112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1715},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5077","reported_seq":5180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.817878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:38.817878+0000","last_peered":"2024-09-15T21:19:38.817878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:38.817878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3628,"num_write_kb":14512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1806}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5131","reported_seq":5241,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.834601+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:38.834601+0000","last_peered":"2024-09-15T21:19:38.834601+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:38.834601+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3530,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5170","reported_seq":5293,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.735079+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:36.735079+0000","last_peered":"2024-09-15T21:19:36.735079+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:36.735079+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5170,"log_dups_size":0,"ondisk_log_size":5170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21168128,"num_objects":5168,"num_object_clones":0,"num_object_copies":10336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2477,"num_write_kb":9908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1735},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5092","reported_seq":5216,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.744821+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:36.744821+0000","last_peered":"2024-09-15T21:19:36.744821+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:36.744821+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5092,"log_dups_size":0,"ondisk_log_size":5092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2677,"num_write_kb":10708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5117","reported_seq":5236,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.768172+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:36.768172+0000","last_peered":"2024-09-15T21:19:36.768172+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:36.768172+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20942848,"num_objects":5113,"num_object_clones":0,"num_object_copies":10226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2764,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1748},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5158","reported_seq":5282,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:36.785538+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:36.785538+0000","last_peered":"2024-09-15T21:19:36.785538+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:36.785538+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5158,"log_dups_size":0,"ondisk_log_size":5158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078633,"num_objects":5147,"num_object_clones":0,"num_object_copies":10294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2818,"num_write_kb":11272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1677},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336405097,"num_objects":82131,"num_object_clones":0,"num_object_copies":164262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":82131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53626,"num_write_kb":214504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":678502400,"data_stored":678418904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82212,"ondisk_log_size":82212,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417812,"kb_used_data":273668,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93954028,"statfs":{"total":96636764160,"available":96208924672,"internally_reserved":0,"allocated":280236032,"data_stored":279980582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147582637},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149728,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453556,"kb_used_data":299748,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93918284,"statfs":{"total":96636764160,"available":96172322816,"internally_reserved":0,"allocated":306941952,"data_stored":306686502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":157478577},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,4,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":187604,"kb_used_data":87876,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94184236,"statfs":{"total":96636764160,"available":96444657664,"internally_reserved":0,"allocated":89985024,"data_stored":89656515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":102100652},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306724864,"data_stored":306721385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91303936,"data_stored":91227398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280473600,"data_stored":280470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:41.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:41.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5058", 'reported_seq': 5130, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.763588+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:36.763588+0000', 'last_peered': '2024-09-15T21:19:36.763588+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:36.763588+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 5058, 'log_dups_size': 0, 'ondisk_log_size': 5058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20717568, 'num_objects': 5058, 'num_object_clones': 0, 'num_object_copies': 10116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5058, 'num_write_kb': 20232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1691}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5202", 'reported_seq': 8810, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:38.759510+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:38.759510+0000', 'last_peered': '2024-09-15T21:19:38.759510+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:38.759510+0000', 'last_undegraded': '2024-09-15T21:19:38.759510+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 5202, 'log_dups_size': 0, 'ondisk_log_size': 5202, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21291008, 'num_objects': 5198, 'num_object_clones': 0, 'num_object_copies': 10396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1647, 'num_objects_unfound': 0, 'num_objects_dirty': 5198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3925, 'num_write_kb': 15700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1746, 'num_bytes_recovered': 7147520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1647}, {'shards': '2', 'objects': 3551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5116", 'reported_seq': 5240, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.743611+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:36.743611+0000', 'last_peered': '2024-09-15T21:19:36.743611+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:36.743611+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 5116, 'log_dups_size': 0, 'ondisk_log_size': 5116, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20934656, 'num_objects': 5111, 'num_object_clones': 0, 'num_object_copies': 10222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2541, 'num_write_kb': 10164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1740}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5240", 'reported_seq': 5358, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.721595+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:36.721595+0000', 'last_peered': '2024-09-15T21:19:36.721595+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:36.721595+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 5240, 'log_dups_size': 0, 'ondisk_log_size': 5240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21438464, 'num_objects': 5234, 'num_object_clones': 0, 'num_object_copies': 10468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2876, 'num_write_kb': 11504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1763}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5091", 'reported_seq': 5210, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:38.743768+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:38.743768+0000', 'last_peered': '2024-09-15T21:19:38.743768+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:38.743768+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 5091, 'log_dups_size': 0, 'ondisk_log_size': 5091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20824064, 'num_objects': 5084, 'num_object_clones': 0, 'num_object_copies': 10168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3677, 'num_write_kb': 14708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1746}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:41.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5064", 'reported_seq': 5182, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.732697+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:36.732697+0000', 'last_peered': '2024-09-15T21:19:36.732697+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:36.732697+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 5064, 'log_dups_size': 0, 'ondisk_log_size': 5064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20709376, 'num_objects': 5056, 'num_object_clones': 0, 'num_object_copies': 10112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2590, 'num_write_kb': 10360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1715}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5077", 'reported_seq': 5180, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:38.817878+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:38.817878+0000', 'last_peered': '2024-09-15T21:19:38.817878+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:38.817878+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 5077, 'log_dups_size': 0, 'ondisk_log_size': 5077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20774912, 'num_objects': 5072, 'num_object_clones': 0, 'num_object_copies': 10144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3628, 'num_write_kb': 14512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1806}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5131", 'reported_seq': 5241, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:38.834601+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:38.834601+0000', 'last_peered': '2024-09-15T21:19:38.834601+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:38.834601+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 5131, 'log_dups_size': 0, 'ondisk_log_size': 5131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20992000, 'num_objects': 5125, 'num_object_clones': 0, 'num_object_copies': 10250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3530, 'num_write_kb': 14120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1775}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5170", 'reported_seq': 5293, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.735079+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:36.735079+0000', 'last_peered': '2024-09-15T21:19:36.735079+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:36.735079+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 5170, 'log_dups_size': 0, 'ondisk_log_size': 5170, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21168128, 'num_objects': 5168, 'num_object_clones': 0, 'num_object_copies': 10336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2477, 'num_write_kb': 9908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1735}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5092", 'reported_seq': 5216, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.744821+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:36.744821+0000', 'last_peered': '2024-09-15T21:19:36.744821+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:36.744821+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 5092, 'log_dups_size': 0, 'ondisk_log_size': 5092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20824064, 'num_objects': 5084, 'num_object_clones': 0, 'num_object_copies': 10168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2677, 'num_write_kb': 10708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1705}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:41.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5117", 'reported_seq': 5236, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.768172+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:36.768172+0000', 'last_peered': '2024-09-15T21:19:36.768172+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:36.768172+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 5117, 'log_dups_size': 0, 'ondisk_log_size': 5117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20942848, 'num_objects': 5113, 'num_object_clones': 0, 'num_object_copies': 10226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2764, 'num_write_kb': 11056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1748}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:41.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5158", 'reported_seq': 5282, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:36.785538+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:36.785538+0000', 'last_peered': '2024-09-15T21:19:36.785538+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:36.785538+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 5158, 'log_dups_size': 0, 'ondisk_log_size': 5158, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21078633, 'num_objects': 5147, 'num_object_clones': 0, 'num_object_copies': 10294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2818, 'num_write_kb': 11272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1677}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:41.643 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:42.488 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 109 20 84847 84827 3.03943 2.23828 0.0186393 0.025694 2024-09-15T21:19:42.893 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:42.894 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/167736 objects degraded (22.161%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":195,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83868,"data_bytes":343511676,"bytes_used":1134592000,"bytes_avail":288775700480,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":167736,"degraded_ratio":0.22161014928220538,"misplaced_objects":1647,"misplaced_total":167736,"misplaced_ratio":0.0098190012877378743,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":149470,"recovering_keys_per_sec":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"write_bytes_sec":2373096,"read_op_per_sec":0,"write_op_per_sec":579},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:19:34.335412+0000","services":{"mon":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:42.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:42.969 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:43.488 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 110 20 85414 85394 3.03193 2.21484 0.0266881 0.0257587 2024-09-15T21:19:43.907 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:43.908 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:43.941 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-15T21:19:42.337256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343511676,"num_objects":83868,"num_object_clones":0,"num_object_copies":167736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":83868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55363,"num_write_kb":221446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1747,"num_bytes_recovered":7147539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83950,"ondisk_log_size":83950,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1108000,"kb_used_data":679792,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":282007520,"statfs":{"total":289910292480,"available":288775700480,"internally_reserved":0,"allocated":696107008,"data_stored":695267599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":438422538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,6,5,5,14,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28483584,"num_objects":6954,"num_object_clones":0,"num_object_copies":13908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-438,"num_objects_unfound":0,"num_objects_dirty":6954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":27816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002707"},"pg_stats":[{"pgid":"2.f","version":"44'5233","reported_seq":5305,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.842691+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:41.842691+0000","last_peered":"2024-09-15T21:19:41.842691+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:41.842691+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":20932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1866},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5217","reported_seq":5308,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.773896+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:38.773896+0000","last_peered":"2024-09-15T21:19:38.773896+0000","last_clean":"2024-09-15T21:19:38.773896+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:38.773896+0000","last_undegraded":"2024-09-15T21:19:38.773896+0000","last_fullsized":"2024-09-15T21:19:38.773896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21344256,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":15772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5202","reported_seq":8810,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:38.759510+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:38.759510+0000","last_peered":"2024-09-15T21:19:38.759510+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:38.759510+0000","last_undegraded":"2024-09-15T21:19:38.759510+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5202,"log_dups_size":0,"ondisk_log_size":5202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21291008,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3925,"num_write_kb":15700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1647},{"shards":"2","objects":3551}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5235","reported_seq":5299,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.815216+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:38.815216+0000","last_peered":"2024-09-15T21:19:38.815216+0000","last_clean":"2024-09-15T21:19:38.815216+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:38.815216+0000","last_undegraded":"2024-09-15T21:19:38.815216+0000","last_fullsized":"2024-09-15T21:19:38.815216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21442560,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5347","reported_seq":5461,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:41.816814+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:41.816814+0000","last_peered":"2024-09-15T21:19:41.816814+0000","last_clean":"2024-09-15T21:19:41.816814+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:41.816814+0000","last_undegraded":"2024-09-15T21:19:41.816814+0000","last_fullsized":"2024-09-15T21:19:41.816814+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5347,"log_dups_size":0,"ondisk_log_size":5347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5297","reported_seq":5418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.810249+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:41.810249+0000","last_peered":"2024-09-15T21:19:41.810249+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:41.810249+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5297,"log_dups_size":0,"ondisk_log_size":5297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2722,"num_write_kb":10888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5406","reported_seq":5524,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.852290+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:41.852290+0000","last_peered":"2024-09-15T21:19:41.852290+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:41.852290+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22118400,"num_objects":5400,"num_object_clones":0,"num_object_copies":10800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1929},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5229","reported_seq":5344,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:41.794582+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:41.794582+0000","last_peered":"2024-09-15T21:19:41.794582+0000","last_clean":"2024-09-15T21:19:41.794582+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:41.794582+0000","last_undegraded":"2024-09-15T21:19:41.794582+0000","last_fullsized":"2024-09-15T21:19:41.794582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5229,"log_dups_size":0,"ondisk_log_size":5229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21409792,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5091","reported_seq":5210,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.743768+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:38.743768+0000","last_peered":"2024-09-15T21:19:38.743768+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:38.743768+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1746}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376347+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376347+0000","last_peered":"2024-09-15T21:18:43.376347+0000","last_clean":"2024-09-15T21:18:43.376347+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376347+0000","last_undegraded":"2024-09-15T21:18:43.376347+0000","last_fullsized":"2024-09-15T21:18:43.376347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5239","reported_seq":5357,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.688098+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:41.688098+0000","last_peered":"2024-09-15T21:19:41.688098+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:41.688098+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2765,"num_write_kb":11060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1890},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5077","reported_seq":5180,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.817878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:38.817878+0000","last_peered":"2024-09-15T21:19:38.817878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:38.817878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3628,"num_write_kb":14512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1806}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5131","reported_seq":5241,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:38.834601+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:38.834601+0000","last_peered":"2024-09-15T21:19:38.834601+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:38.834601+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992000,"num_objects":5125,"num_object_clones":0,"num_object_copies":10250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3530,"num_write_kb":14120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5335","reported_seq":5458,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.838094+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:41.838094+0000","last_peered":"2024-09-15T21:19:41.838094+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:41.838094+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2642,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1900},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5280","reported_seq":5404,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.841191+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:41.841191+0000","last_peered":"2024-09-15T21:19:41.841191+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:41.841191+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5280,"log_dups_size":0,"ondisk_log_size":5280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21594112,"num_objects":5272,"num_object_clones":0,"num_object_copies":10544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":11460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1893},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439422+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439422+0000","last_peered":"2024-09-15T21:18:43.439422+0000","last_clean":"2024-09-15T21:18:43.439422+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439422+0000","last_undegraded":"2024-09-15T21:18:43.439422+0000","last_fullsized":"2024-09-15T21:18:43.439422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5279","reported_seq":5398,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.794099+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:41.794099+0000","last_peered":"2024-09-15T21:19:41.794099+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:41.794099+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5279,"log_dups_size":0,"ondisk_log_size":5279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2926,"num_write_kb":11704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1910},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5349","reported_seq":5473,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.820211+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:41.820211+0000","last_peered":"2024-09-15T21:19:41.820211+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:41.820211+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860969,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3009,"num_write_kb":12036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1868},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343511657,"num_objects":83866,"num_object_clones":0,"num_object_copies":167732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1647,"num_objects_unfound":0,"num_objects_dirty":83866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55361,"num_write_kb":221444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696004608,"data_stored":695921112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83947,"ondisk_log_size":83947,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953501,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417812,"kb_used_data":273668,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93954028,"statfs":{"total":96636764160,"available":96208924672,"internally_reserved":0,"allocated":280236032,"data_stored":279980582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":147582637},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149729,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465912,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93905928,"statfs":{"total":96636764160,"available":96159670272,"internally_reserved":0,"allocated":315334656,"data_stored":315079206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":161738417},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,4,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":224276,"kb_used_data":98180,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94147564,"statfs":{"total":96636764160,"available":96407105536,"internally_reserved":0,"allocated":100536320,"data_stored":100207811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":129101484},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315191296,"data_stored":315187817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100339712,"data_stored":100263174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280473600,"data_stored":280470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:44.488 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 111 20 85970 85950 3.02419 2.17188 0.0130584 0.0258227 2024-09-15T21:19:45.490 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 112 20 86571 86551 3.0181 2.34766 0.0203461 0.025876 2024-09-15T21:19:46.490 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 113 20 87180 87160 3.01245 2.37891 0.0619963 0.0259244 2024-09-15T21:19:46.944 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:47.490 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 114 20 87769 87749 3.0062 2.30078 0.046934 0.0259802 2024-09-15T21:19:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:47.906 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:47.944 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-15T21:19:46.338229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":347787900,"num_objects":84912,"num_object_clones":0,"num_object_copies":169824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":84912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56407,"num_write_kb":225622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1901,"num_bytes_recovered":7778323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84994,"ondisk_log_size":84994,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1117500,"kb_used_data":685132,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":281998020,"statfs":{"total":289910292480,"available":288765972480,"internally_reserved":0,"allocated":701575168,"data_stored":700735759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":442682378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,6,5,5,14,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-342,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1400832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002893"},"pg_stats":[{"pgid":"2.f","version":"44'5233","reported_seq":5305,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.842691+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:41.842691+0000","last_peered":"2024-09-15T21:19:41.842691+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:41.842691+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":20932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1866},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5381","reported_seq":5472,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:43.811024+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:43.811024+0000","last_peered":"2024-09-15T21:19:43.811024+0000","last_clean":"2024-09-15T21:19:43.811024+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:43.811024+0000","last_undegraded":"2024-09-15T21:19:43.811024+0000","last_fullsized":"2024-09-15T21:19:43.811024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22016000,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4107,"num_write_kb":16428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5360","reported_seq":9276,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:43.843739+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:43.843739+0000","last_peered":"2024-09-15T21:19:43.843739+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:43.843739+0000","last_undegraded":"2024-09-15T21:19:43.843739+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5360,"log_dups_size":0,"ondisk_log_size":5360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4083,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1493},{"shards":"2","objects":3863}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5405","reported_seq":5469,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:43.788195+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:43.788195+0000","last_peered":"2024-09-15T21:19:43.788195+0000","last_clean":"2024-09-15T21:19:43.788195+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:43.788195+0000","last_undegraded":"2024-09-15T21:19:43.788195+0000","last_fullsized":"2024-09-15T21:19:43.788195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5405,"num_write_kb":21620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5347","reported_seq":5461,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:41.816814+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:41.816814+0000","last_peered":"2024-09-15T21:19:41.816814+0000","last_clean":"2024-09-15T21:19:41.816814+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:41.816814+0000","last_undegraded":"2024-09-15T21:19:41.816814+0000","last_fullsized":"2024-09-15T21:19:41.816814+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5347,"log_dups_size":0,"ondisk_log_size":5347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5297","reported_seq":5418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.810249+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:41.810249+0000","last_peered":"2024-09-15T21:19:41.810249+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:41.810249+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5297,"log_dups_size":0,"ondisk_log_size":5297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2722,"num_write_kb":10888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5406","reported_seq":5524,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.852290+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:41.852290+0000","last_peered":"2024-09-15T21:19:41.852290+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:41.852290+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22118400,"num_objects":5400,"num_object_clones":0,"num_object_copies":10800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1929},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5229","reported_seq":5344,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:41.794582+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:41.794582+0000","last_peered":"2024-09-15T21:19:41.794582+0000","last_clean":"2024-09-15T21:19:41.794582+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:41.794582+0000","last_undegraded":"2024-09-15T21:19:41.794582+0000","last_fullsized":"2024-09-15T21:19:41.794582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5229,"log_dups_size":0,"ondisk_log_size":5229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21409792,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5299","reported_seq":5418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.848050+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:43.848050+0000","last_peered":"2024-09-15T21:19:43.848050+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:43.848050+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5239","reported_seq":5357,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.688098+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:41.688098+0000","last_peered":"2024-09-15T21:19:41.688098+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:41.688098+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2765,"num_write_kb":11060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1890},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5250","reported_seq":5355,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.857503+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:43.857503+0000","last_peered":"2024-09-15T21:19:43.857503+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:43.857503+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3801,"num_write_kb":15204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1979}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5302","reported_seq":5411,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.848383+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:43.848383+0000","last_peered":"2024-09-15T21:19:43.848383+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:43.848383+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3701,"num_write_kb":14804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1946}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5335","reported_seq":5458,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.838094+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:41.838094+0000","last_peered":"2024-09-15T21:19:41.838094+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:41.838094+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2642,"num_write_kb":10568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1900},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5280","reported_seq":5404,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.841191+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:41.841191+0000","last_peered":"2024-09-15T21:19:41.841191+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:41.841191+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5280,"log_dups_size":0,"ondisk_log_size":5280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21594112,"num_objects":5272,"num_object_clones":0,"num_object_copies":10544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":11460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1893},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5279","reported_seq":5398,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.794099+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:41.794099+0000","last_peered":"2024-09-15T21:19:41.794099+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:41.794099+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5279,"log_dups_size":0,"ondisk_log_size":5279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2926,"num_write_kb":11704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1910},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5349","reported_seq":5473,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:41.820211+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:41.820211+0000","last_peered":"2024-09-15T21:19:41.820211+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:41.820211+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21860969,"num_objects":5338,"num_object_clones":0,"num_object_copies":10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3009,"num_write_kb":12036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1868},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":347787881,"num_objects":84910,"num_object_clones":0,"num_object_copies":169820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":84910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56405,"num_write_kb":225620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":701685760,"data_stored":701602264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84991,"ondisk_log_size":84991,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953502,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427312,"kb_used_data":279008,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944528,"statfs":{"total":96636764160,"available":96199196672,"internally_reserved":0,"allocated":285704192,"data_stored":285448742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149729,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465912,"kb_used_data":307944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93905928,"statfs":{"total":96636764160,"available":96159670272,"internally_reserved":0,"allocated":315334656,"data_stored":315079206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":161738417},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,4,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":224276,"kb_used_data":98180,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94147564,"statfs":{"total":96636764160,"available":96407105536,"internally_reserved":0,"allocated":100536320,"data_stored":100207811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":129101484},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315191296,"data_stored":315187817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100339712,"data_stored":100263174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286154752,"data_stored":286151273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:47.946 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:48.490 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 115 20 88333 88313 2.99922 2.20312 0.015833 0.0260384 2024-09-15T21:19:48.807 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:48.809 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:48.903 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-15T21:19:48.338670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355295868,"num_objects":86745,"num_object_clones":0,"num_object_copies":173490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":86745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58240,"num_write_kb":232954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1901,"num_bytes_recovered":7778323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86827,"ondisk_log_size":86827,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1153708,"kb_used_data":703292,"kb_used_omap":12,"kb_used_meta":450355,"kb_avail":281961812,"statfs":{"total":289910292480,"available":288728895488,"internally_reserved":0,"allocated":720171008,"data_stored":719331599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":461163530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,6,5,9,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-342,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7166,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1400832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002910"},"pg_stats":[{"pgid":"2.f","version":"44'5407","reported_seq":5479,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.805952+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:46.805952+0000","last_peered":"2024-09-15T21:19:46.805952+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:46.805952+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5407,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2040},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5381","reported_seq":5472,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:43.811024+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:43.811024+0000","last_peered":"2024-09-15T21:19:43.811024+0000","last_clean":"2024-09-15T21:19:43.811024+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:43.811024+0000","last_undegraded":"2024-09-15T21:19:43.811024+0000","last_fullsized":"2024-09-15T21:19:43.811024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22016000,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4107,"num_write_kb":16428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5360","reported_seq":9276,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:43.843739+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:43.843739+0000","last_peered":"2024-09-15T21:19:43.843739+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:43.843739+0000","last_undegraded":"2024-09-15T21:19:43.843739+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5360,"log_dups_size":0,"ondisk_log_size":5360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4083,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1493},{"shards":"2","objects":3863}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5405","reported_seq":5469,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:43.788195+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:43.788195+0000","last_peered":"2024-09-15T21:19:43.788195+0000","last_clean":"2024-09-15T21:19:43.788195+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:43.788195+0000","last_undegraded":"2024-09-15T21:19:43.788195+0000","last_fullsized":"2024-09-15T21:19:43.788195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5405,"num_write_kb":21620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5518","reported_seq":5632,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:46.802160+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:46.802160+0000","last_peered":"2024-09-15T21:19:46.802160+0000","last_clean":"2024-09-15T21:19:46.802160+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:46.802160+0000","last_undegraded":"2024-09-15T21:19:46.802160+0000","last_fullsized":"2024-09-15T21:19:46.802160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5456","reported_seq":5581,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.826879+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:46.826879+0000","last_peered":"2024-09-15T21:19:46.826879+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:46.826879+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22327296,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2881,"num_write_kb":11524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2080},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5609","reported_seq":5724,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.838120+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:46.838120+0000","last_peered":"2024-09-15T21:19:46.838120+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:46.838120+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5609,"log_dups_size":0,"ondisk_log_size":5609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22949888,"num_objects":5603,"num_object_clones":0,"num_object_copies":11206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3245,"num_write_kb":12980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2132},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5403","reported_seq":5518,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:46.688283+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:46.688283+0000","last_peered":"2024-09-15T21:19:46.688283+0000","last_clean":"2024-09-15T21:19:46.688283+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:46.688283+0000","last_undegraded":"2024-09-15T21:19:46.688283+0000","last_fullsized":"2024-09-15T21:19:46.688283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5403,"log_dups_size":0,"ondisk_log_size":5403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22122496,"num_objects":5401,"num_object_clones":0,"num_object_copies":10802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5299","reported_seq":5418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.848050+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:43.848050+0000","last_peered":"2024-09-15T21:19:43.848050+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:43.848050+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":1954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5414","reported_seq":5532,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.864868+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:46.864868+0000","last_peered":"2024-09-15T21:19:46.864868+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:46.864868+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5414,"log_dups_size":0,"ondisk_log_size":5414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2940,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5250","reported_seq":5355,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.857503+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:43.857503+0000","last_peered":"2024-09-15T21:19:43.857503+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:43.857503+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3801,"num_write_kb":15204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":1979}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5302","reported_seq":5411,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:43.848383+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:43.848383+0000","last_peered":"2024-09-15T21:19:43.848383+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:43.848383+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3701,"num_write_kb":14804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":1946}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5536","reported_seq":5658,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.804323+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:46.804323+0000","last_peered":"2024-09-15T21:19:46.804323+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:46.804323+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5536,"log_dups_size":0,"ondisk_log_size":5536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22667264,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2843,"num_write_kb":11372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5470","reported_seq":5595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.800918+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:46.800918+0000","last_peered":"2024-09-15T21:19:46.800918+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:46.800918+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5470,"log_dups_size":0,"ondisk_log_size":5470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22372352,"num_objects":5462,"num_object_clones":0,"num_object_copies":10924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2083},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5470","reported_seq":5589,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.849117+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:46.849117+0000","last_peered":"2024-09-15T21:19:46.849117+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:46.849117+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5470,"log_dups_size":0,"ondisk_log_size":5470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3117,"num_write_kb":12468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5544","reported_seq":5668,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.816131+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:46.816131+0000","last_peered":"2024-09-15T21:19:46.816131+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:46.816131+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5544,"log_dups_size":0,"ondisk_log_size":5544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22659689,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":355295849,"num_objects":86743,"num_object_clones":0,"num_object_copies":173486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1493,"num_objects_unfound":0,"num_objects_dirty":86743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58238,"num_write_kb":232952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":720388096,"data_stored":720304600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86824,"ondisk_log_size":86824,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953502,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427312,"kb_used_data":279008,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944528,"statfs":{"total":96636764160,"available":96199196672,"internally_reserved":0,"allocated":285704192,"data_stored":285448742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482892,"kb_used_data":316604,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93888948,"statfs":{"total":96636764160,"available":96142282752,"internally_reserved":0,"allocated":324202496,"data_stored":323947046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":170258097},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243504,"kb_used_data":107680,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94128336,"statfs":{"total":96636764160,"available":96387416064,"internally_reserved":0,"allocated":110264320,"data_stored":109935811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":139062956},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324116480,"data_stored":324113001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110116864,"data_stored":110040326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286154752,"data_stored":286151273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:48.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:48.904 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5407", 'reported_seq': 5479, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.805952+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:46.805952+0000', 'last_peered': '2024-09-15T21:19:46.805952+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:46.805952+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 5407, 'log_dups_size': 0, 'ondisk_log_size': 5407, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22147072, 'num_objects': 5407, 'num_object_clones': 0, 'num_object_copies': 10814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5407, 'num_write_kb': 21628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2040}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5360", 'reported_seq': 9276, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:43.843739+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:43.843739+0000', 'last_peered': '2024-09-15T21:19:43.843739+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:43.843739+0000', 'last_undegraded': '2024-09-15T21:19:43.843739+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 5360, 'log_dups_size': 0, 'ondisk_log_size': 5360, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21938176, 'num_objects': 5356, 'num_object_clones': 0, 'num_object_copies': 10712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1493, 'num_objects_unfound': 0, 'num_objects_dirty': 5356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4083, 'num_write_kb': 16332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1900, 'num_bytes_recovered': 7778304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1493}, {'shards': '2', 'objects': 3863}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5456", 'reported_seq': 5581, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.826879+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:46.826879+0000', 'last_peered': '2024-09-15T21:19:46.826879+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:46.826879+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 5456, 'log_dups_size': 0, 'ondisk_log_size': 5456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22327296, 'num_objects': 5451, 'num_object_clones': 0, 'num_object_copies': 10902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2881, 'num_write_kb': 11524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2080}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5609", 'reported_seq': 5724, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.838120+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:46.838120+0000', 'last_peered': '2024-09-15T21:19:46.838120+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:46.838120+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 5609, 'log_dups_size': 0, 'ondisk_log_size': 5609, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22949888, 'num_objects': 5603, 'num_object_clones': 0, 'num_object_copies': 11206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5603, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3245, 'num_write_kb': 12980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2132}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5299", 'reported_seq': 5418, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:43.848050+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:43.848050+0000', 'last_peered': '2024-09-15T21:19:43.848050+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:43.848050+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 5299, 'log_dups_size': 0, 'ondisk_log_size': 5299, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21676032, 'num_objects': 5292, 'num_object_clones': 0, 'num_object_copies': 10584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3885, 'num_write_kb': 15540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 1954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:48.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5414", 'reported_seq': 5532, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.864868+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:46.864868+0000', 'last_peered': '2024-09-15T21:19:46.864868+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:46.864868+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 5414, 'log_dups_size': 0, 'ondisk_log_size': 5414, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22142976, 'num_objects': 5406, 'num_object_clones': 0, 'num_object_copies': 10812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2940, 'num_write_kb': 11760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2065}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5250", 'reported_seq': 5355, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:43.857503+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:43.857503+0000', 'last_peered': '2024-09-15T21:19:43.857503+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:43.857503+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 5250, 'log_dups_size': 0, 'ondisk_log_size': 5250, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21483520, 'num_objects': 5245, 'num_object_clones': 0, 'num_object_copies': 10490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5245, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3801, 'num_write_kb': 15204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 1979}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5302", 'reported_seq': 5411, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:43.848383+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:43.848383+0000', 'last_peered': '2024-09-15T21:19:43.848383+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:43.848383+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 5302, 'log_dups_size': 0, 'ondisk_log_size': 5302, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21692416, 'num_objects': 5296, 'num_object_clones': 0, 'num_object_copies': 10592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3701, 'num_write_kb': 14804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 1946}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:48.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5536", 'reported_seq': 5658, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.804323+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:46.804323+0000', 'last_peered': '2024-09-15T21:19:46.804323+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:46.804323+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 5536, 'log_dups_size': 0, 'ondisk_log_size': 5536, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22667264, 'num_objects': 5534, 'num_object_clones': 0, 'num_object_copies': 11068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2843, 'num_write_kb': 11372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2101}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5470", 'reported_seq': 5595, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.800918+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:46.800918+0000', 'last_peered': '2024-09-15T21:19:46.800918+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:46.800918+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 5470, 'log_dups_size': 0, 'ondisk_log_size': 5470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22372352, 'num_objects': 5462, 'num_object_clones': 0, 'num_object_copies': 10924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3055, 'num_write_kb': 12220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2083}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5470", 'reported_seq': 5589, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.849117+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:46.849117+0000', 'last_peered': '2024-09-15T21:19:46.849117+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:46.849117+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 5470, 'log_dups_size': 0, 'ondisk_log_size': 5470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22388736, 'num_objects': 5466, 'num_object_clones': 0, 'num_object_copies': 10932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3117, 'num_write_kb': 12468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2101}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:48.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5544", 'reported_seq': 5668, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:46.816131+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:46.816131+0000', 'last_peered': '2024-09-15T21:19:46.816131+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:46.816131+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 5544, 'log_dups_size': 0, 'ondisk_log_size': 5544, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22659689, 'num_objects': 5533, 'num_object_clones': 0, 'num_object_copies': 11066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3204, 'num_write_kb': 12816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2063}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:48.908 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:49.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 116 20 88890 88870 2.99207 2.17578 0.0332286 0.0261027 2024-09-15T21:19:49.984 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:49.984 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/173490 objects degraded (21.426%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":203,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86745,"data_bytes":355295868,"bytes_used":1181396992,"bytes_avail":288728895488,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":173490,"degraded_ratio":0.21426018790708398,"misplaced_objects":1493,"misplaced_total":173490,"misplaced_ratio":0.0086056833246873025,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116707,"recovering_keys_per_sec":0,"num_objects_recovered":342,"num_bytes_recovered":1400832,"num_keys_recovered":0,"write_bytes_sec":2445401,"read_op_per_sec":0,"write_op_per_sec":597},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:19:44.337807+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:50.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:50.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:50.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 117 20 89461 89441 2.98556 2.23047 0.0113926 0.0261572 2024-09-15T21:19:50.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:50.796 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:50.828 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-15T21:19:50.339169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":359715452,"num_objects":87824,"num_object_clones":0,"num_object_copies":175648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1329,"num_objects_unfound":0,"num_objects_dirty":87824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59319,"num_write_kb":237270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2065,"num_bytes_recovered":8450067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87906,"ondisk_log_size":87906,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1146968,"kb_used_data":709032,"kb_used_omap":12,"kb_used_meta":437875,"kb_avail":281968552,"statfs":{"total":289910292480,"available":288735797248,"internally_reserved":0,"allocated":726048768,"data_stored":725209359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":448384010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,8,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27246592,"num_objects":6652,"num_object_clones":0,"num_object_copies":13304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-506,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":2072576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002974"},"pg_stats":[{"pgid":"2.f","version":"44'5407","reported_seq":5479,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.805952+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:46.805952+0000","last_peered":"2024-09-15T21:19:46.805952+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:46.805952+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5407,"log_dups_size":0,"ondisk_log_size":5407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5407,"num_write_kb":21628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2040},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5568","reported_seq":5659,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:48.866185+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:48.866185+0000","last_peered":"2024-09-15T21:19:48.866185+0000","last_clean":"2024-09-15T21:19:48.866185+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:48.866185+0000","last_undegraded":"2024-09-15T21:19:48.866185+0000","last_fullsized":"2024-09-15T21:19:48.866185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5568,"log_dups_size":0,"ondisk_log_size":5568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22781952,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4294,"num_write_kb":17176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5548","reported_seq":9792,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:48.869091+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:48.869091+0000","last_peered":"2024-09-15T21:19:48.869091+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:48.869091+0000","last_undegraded":"2024-09-15T21:19:48.869091+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1329,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8450048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1329},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5576","reported_seq":5640,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:48.823066+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:48.823066+0000","last_peered":"2024-09-15T21:19:48.823066+0000","last_clean":"2024-09-15T21:19:48.823066+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:48.823066+0000","last_undegraded":"2024-09-15T21:19:48.823066+0000","last_fullsized":"2024-09-15T21:19:48.823066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5576,"log_dups_size":0,"ondisk_log_size":5576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22839296,"num_objects":5576,"num_object_clones":0,"num_object_copies":11152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5518","reported_seq":5632,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:46.802160+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:46.802160+0000","last_peered":"2024-09-15T21:19:46.802160+0000","last_clean":"2024-09-15T21:19:46.802160+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:46.802160+0000","last_undegraded":"2024-09-15T21:19:46.802160+0000","last_fullsized":"2024-09-15T21:19:46.802160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3376,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5456","reported_seq":5581,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.826879+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:46.826879+0000","last_peered":"2024-09-15T21:19:46.826879+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:46.826879+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22327296,"num_objects":5451,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2881,"num_write_kb":11524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2080},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5609","reported_seq":5724,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.838120+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:46.838120+0000","last_peered":"2024-09-15T21:19:46.838120+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:46.838120+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5609,"log_dups_size":0,"ondisk_log_size":5609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22949888,"num_objects":5603,"num_object_clones":0,"num_object_copies":11206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3245,"num_write_kb":12980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2132},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5403","reported_seq":5518,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:46.688283+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:46.688283+0000","last_peered":"2024-09-15T21:19:46.688283+0000","last_clean":"2024-09-15T21:19:46.688283+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:46.688283+0000","last_undegraded":"2024-09-15T21:19:46.688283+0000","last_fullsized":"2024-09-15T21:19:46.688283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5403,"log_dups_size":0,"ondisk_log_size":5403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22122496,"num_objects":5401,"num_object_clones":0,"num_object_copies":10802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5475","reported_seq":5590,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:48.819059+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:48.819059+0000","last_peered":"2024-09-15T21:19:48.819059+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:48.819059+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22396928,"num_objects":5468,"num_object_clones":0,"num_object_copies":10936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5414","reported_seq":5532,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.864868+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:46.864868+0000","last_peered":"2024-09-15T21:19:46.864868+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:46.864868+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5414,"log_dups_size":0,"ondisk_log_size":5414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22142976,"num_objects":5406,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2940,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2065},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5424","reported_seq":5527,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:48.846881+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:48.846881+0000","last_peered":"2024-09-15T21:19:48.846881+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:48.846881+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5424,"log_dups_size":0,"ondisk_log_size":5424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22196224,"num_objects":5419,"num_object_clones":0,"num_object_copies":10838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3975,"num_write_kb":15900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5485","reported_seq":5595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:48.808965+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:48.808965+0000","last_peered":"2024-09-15T21:19:48.808965+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:48.808965+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5485,"log_dups_size":0,"ondisk_log_size":5485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22441984,"num_objects":5479,"num_object_clones":0,"num_object_copies":10958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3884,"num_write_kb":15536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2129}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5536","reported_seq":5658,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.804323+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:46.804323+0000","last_peered":"2024-09-15T21:19:46.804323+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:46.804323+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5536,"log_dups_size":0,"ondisk_log_size":5536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22667264,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2843,"num_write_kb":11372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5470","reported_seq":5595,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.800918+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:46.800918+0000","last_peered":"2024-09-15T21:19:46.800918+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:46.800918+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5470,"log_dups_size":0,"ondisk_log_size":5470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22372352,"num_objects":5462,"num_object_clones":0,"num_object_copies":10924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2083},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5470","reported_seq":5589,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.849117+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:46.849117+0000","last_peered":"2024-09-15T21:19:46.849117+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:46.849117+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5470,"log_dups_size":0,"ondisk_log_size":5470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22388736,"num_objects":5466,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3117,"num_write_kb":12468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5544","reported_seq":5668,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:46.816131+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:46.816131+0000","last_peered":"2024-09-15T21:19:46.816131+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:46.816131+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5544,"log_dups_size":0,"ondisk_log_size":5544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22659689,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3204,"num_write_kb":12816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":359715433,"num_objects":87822,"num_object_clones":0,"num_object_copies":175644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1329,"num_objects_unfound":0,"num_objects_dirty":87822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59317,"num_write_kb":237268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2064,"num_bytes_recovered":8450048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":726278144,"data_stored":726194648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87903,"ondisk_log_size":87903,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420572,"kb_used_data":284748,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93951268,"statfs":{"total":96636764160,"available":96206098432,"internally_reserved":0,"allocated":291581952,"data_stored":291326502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":139062957},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149730,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482892,"kb_used_data":316604,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93888948,"statfs":{"total":96636764160,"available":96142282752,"internally_reserved":0,"allocated":324202496,"data_stored":323947046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":170258097},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243504,"kb_used_data":107680,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94128336,"statfs":{"total":96636764160,"available":96387416064,"internally_reserved":0,"allocated":110264320,"data_stored":109935811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":139062956},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324116480,"data_stored":324113001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110116864,"data_stored":110040326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292044800,"data_stored":292041321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:51.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 118 20 89986 89966 2.97764 2.05078 0.0224846 0.0262297 2024-09-15T21:19:52.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 119 20 90590 90570 2.97244 2.35938 0.0368704 0.0262738 2024-09-15T21:19:53.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:19:53.490940+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0263214 2024-09-15T21:19:53.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:19:53.492 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 120 20 91182 91162 2.96694 2.3125 0.0428012 0.0263214 2024-09-15T21:19:53.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:54.495 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 121 20 91739 91719 2.96038 2.17578 0.0324846 0.0263798 2024-09-15T21:19:54.680 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:54.681 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:54.721 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-15T21:19:54.346279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371335804,"num_objects":90661,"num_object_clones":0,"num_object_copies":181322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62156,"num_write_kb":248618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":9048083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90743,"ondisk_log_size":90743,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1192748,"kb_used_data":732604,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281922772,"statfs":{"total":289910292480,"available":288688918528,"internally_reserved":0,"allocated":750186496,"data_stored":749347087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":471125002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,5,5,16,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27824128,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-464,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6793,"num_write_kb":27172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009023"},"pg_stats":[{"pgid":"2.f","version":"44'5570","reported_seq":5642,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.858631+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:51.858631+0000","last_peered":"2024-09-15T21:19:51.858631+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:51.858631+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5570,"num_write_kb":22280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2203},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5737","reported_seq":5828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865979+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:53.865979+0000","last_peered":"2024-09-15T21:19:53.865979+0000","last_clean":"2024-09-15T21:19:53.865979+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:53.865979+0000","last_undegraded":"2024-09-15T21:19:53.865979+0000","last_fullsized":"2024-09-15T21:19:53.865979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5726","reported_seq":10262,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:53.878062+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:53.878062+0000","last_peered":"2024-09-15T21:19:53.878062+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:53.878062+0000","last_undegraded":"2024-09-15T21:19:53.878062+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4449,"num_write_kb":17796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1183},{"shards":"2","objects":4539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5767","reported_seq":5832,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865474+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:53.865474+0000","last_peered":"2024-09-15T21:19:53.865474+0000","last_clean":"2024-09-15T21:19:53.865474+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:53.865474+0000","last_undegraded":"2024-09-15T21:19:53.865474+0000","last_fullsized":"2024-09-15T21:19:53.865474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5767,"num_write_kb":23068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5693","reported_seq":5807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.779592+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:51.779592+0000","last_peered":"2024-09-15T21:19:51.779592+0000","last_clean":"2024-09-15T21:19:51.779592+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:51.779592+0000","last_undegraded":"2024-09-15T21:19:51.779592+0000","last_fullsized":"2024-09-15T21:19:51.779592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3551,"num_write_kb":14204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5641","reported_seq":5766,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.785533+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:51.785533+0000","last_peered":"2024-09-15T21:19:51.785533+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:51.785533+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3066,"num_write_kb":12264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5796","reported_seq":5909,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.856691+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:51.856691+0000","last_peered":"2024-09-15T21:19:51.856691+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:51.856691+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3432,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5592","reported_seq":5707,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.858777+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:51.858777+0000","last_peered":"2024-09-15T21:19:51.858777+0000","last_clean":"2024-09-15T21:19:51.858777+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:51.858777+0000","last_undegraded":"2024-09-15T21:19:51.858777+0000","last_fullsized":"2024-09-15T21:19:51.858777+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22896640,"num_objects":5590,"num_object_clones":0,"num_object_copies":11180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5641","reported_seq":5760,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.820391+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:53.820391+0000","last_peered":"2024-09-15T21:19:53.820391+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:53.820391+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5574","reported_seq":5692,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.783711+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:51.783711+0000","last_peered":"2024-09-15T21:19:51.783711+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:51.783711+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2225},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5624","reported_seq":5729,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.837498+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:53.837498+0000","last_peered":"2024-09-15T21:19:53.837498+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:53.837498+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5624,"log_dups_size":0,"ondisk_log_size":5624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23015424,"num_objects":5619,"num_object_clones":0,"num_object_copies":11238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4175,"num_write_kb":16700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2353}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5693","reported_seq":5803,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.831015+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:53.831015+0000","last_peered":"2024-09-15T21:19:53.831015+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:53.831015+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4092,"num_write_kb":16368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5690","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.812247+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:51.812247+0000","last_peered":"2024-09-15T21:19:51.812247+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:51.812247+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5639","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.874977+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:51.874977+0000","last_peered":"2024-09-15T21:19:51.874977+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:51.874977+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23064576,"num_objects":5631,"num_object_clones":0,"num_object_copies":11262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":12896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2252},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5653","reported_seq":5772,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.818724+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:51.818724+0000","last_peered":"2024-09-15T21:19:51.818724+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:51.818724+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5653,"log_dups_size":0,"ondisk_log_size":5653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5704","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.741818+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:51.741818+0000","last_peered":"2024-09-15T21:19:51.741818+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:51.741818+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23315049,"num_objects":5693,"num_object_clones":0,"num_object_copies":11386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3364,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2223},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371335785,"num_objects":90659,"num_object_clones":0,"num_object_copies":181318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62154,"num_write_kb":248616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750120960,"data_stored":750037464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90740,"ondisk_log_size":90740,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430816,"kb_used_data":290832,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93941024,"statfs":{"total":96636764160,"available":96195608576,"internally_reserved":0,"allocated":297811968,"data_stored":297556518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500864,"kb_used_data":324848,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870976,"statfs":{"total":96636764160,"available":96123879424,"internally_reserved":0,"allocated":332644352,"data_stored":332388902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261068,"kb_used_data":116924,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94110772,"statfs":{"total":96636764160,"available":96369430528,"internally_reserved":0,"allocated":119730176,"data_stored":119401667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332623872,"data_stored":332620393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119410688,"data_stored":119334150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298086400,"data_stored":298082921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:54.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:55.493 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 122 20 92276 92256 2.95331 2.09766 0.0518101 0.0264422 2024-09-15T21:19:55.560 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:55.615 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-15T21:19:54.346279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371335804,"num_objects":90661,"num_object_clones":0,"num_object_copies":181322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62156,"num_write_kb":248618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":9048083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90743,"ondisk_log_size":90743,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1192748,"kb_used_data":732604,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281922772,"statfs":{"total":289910292480,"available":288688918528,"internally_reserved":0,"allocated":750186496,"data_stored":749347087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":471125002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,5,5,16,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27824128,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-464,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6793,"num_write_kb":27172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1900544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009023"},"pg_stats":[{"pgid":"2.f","version":"44'5570","reported_seq":5642,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.858631+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:51.858631+0000","last_peered":"2024-09-15T21:19:51.858631+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:51.858631+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5570,"num_write_kb":22280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2203},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5737","reported_seq":5828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865979+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:53.865979+0000","last_peered":"2024-09-15T21:19:53.865979+0000","last_clean":"2024-09-15T21:19:53.865979+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:53.865979+0000","last_undegraded":"2024-09-15T21:19:53.865979+0000","last_fullsized":"2024-09-15T21:19:53.865979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5726","reported_seq":10262,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:53.878062+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:53.878062+0000","last_peered":"2024-09-15T21:19:53.878062+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:53.878062+0000","last_undegraded":"2024-09-15T21:19:53.878062+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4449,"num_write_kb":17796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1183},{"shards":"2","objects":4539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5767","reported_seq":5832,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865474+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:53.865474+0000","last_peered":"2024-09-15T21:19:53.865474+0000","last_clean":"2024-09-15T21:19:53.865474+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:53.865474+0000","last_undegraded":"2024-09-15T21:19:53.865474+0000","last_fullsized":"2024-09-15T21:19:53.865474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5767,"num_write_kb":23068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5693","reported_seq":5807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.779592+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:51.779592+0000","last_peered":"2024-09-15T21:19:51.779592+0000","last_clean":"2024-09-15T21:19:51.779592+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:51.779592+0000","last_undegraded":"2024-09-15T21:19:51.779592+0000","last_fullsized":"2024-09-15T21:19:51.779592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3551,"num_write_kb":14204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5641","reported_seq":5766,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.785533+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:51.785533+0000","last_peered":"2024-09-15T21:19:51.785533+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:51.785533+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3066,"num_write_kb":12264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5796","reported_seq":5909,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.856691+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:51.856691+0000","last_peered":"2024-09-15T21:19:51.856691+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:51.856691+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3432,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5592","reported_seq":5707,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.858777+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:51.858777+0000","last_peered":"2024-09-15T21:19:51.858777+0000","last_clean":"2024-09-15T21:19:51.858777+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:51.858777+0000","last_undegraded":"2024-09-15T21:19:51.858777+0000","last_fullsized":"2024-09-15T21:19:51.858777+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22896640,"num_objects":5590,"num_object_clones":0,"num_object_copies":11180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5641","reported_seq":5760,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.820391+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:53.820391+0000","last_peered":"2024-09-15T21:19:53.820391+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:53.820391+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5574","reported_seq":5692,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.783711+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:51.783711+0000","last_peered":"2024-09-15T21:19:51.783711+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:51.783711+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2225},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5624","reported_seq":5729,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.837498+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:53.837498+0000","last_peered":"2024-09-15T21:19:53.837498+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:53.837498+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5624,"log_dups_size":0,"ondisk_log_size":5624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23015424,"num_objects":5619,"num_object_clones":0,"num_object_copies":11238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4175,"num_write_kb":16700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2353}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5693","reported_seq":5803,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.831015+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:53.831015+0000","last_peered":"2024-09-15T21:19:53.831015+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:53.831015+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4092,"num_write_kb":16368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5690","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.812247+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:51.812247+0000","last_peered":"2024-09-15T21:19:51.812247+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:51.812247+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5639","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.874977+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:51.874977+0000","last_peered":"2024-09-15T21:19:51.874977+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:51.874977+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23064576,"num_objects":5631,"num_object_clones":0,"num_object_copies":11262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":12896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2252},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5653","reported_seq":5772,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.818724+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:51.818724+0000","last_peered":"2024-09-15T21:19:51.818724+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:51.818724+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5653,"log_dups_size":0,"ondisk_log_size":5653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5704","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.741818+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:51.741818+0000","last_peered":"2024-09-15T21:19:51.741818+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:51.741818+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23315049,"num_objects":5693,"num_object_clones":0,"num_object_copies":11386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3364,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2223},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371335785,"num_objects":90659,"num_object_clones":0,"num_object_copies":181318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62154,"num_write_kb":248616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750120960,"data_stored":750037464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90740,"ondisk_log_size":90740,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430816,"kb_used_data":290832,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93941024,"statfs":{"total":96636764160,"available":96195608576,"internally_reserved":0,"allocated":297811968,"data_stored":297556518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500864,"kb_used_data":324848,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870976,"statfs":{"total":96636764160,"available":96123879424,"internally_reserved":0,"allocated":332644352,"data_stored":332388902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261068,"kb_used_data":116924,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94110772,"statfs":{"total":96636764160,"available":96369430528,"internally_reserved":0,"allocated":119730176,"data_stored":119401667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332623872,"data_stored":332620393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119410688,"data_stored":119334150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298086400,"data_stored":298082921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:55.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:19:55.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5570", 'reported_seq': 5642, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.858631+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:51.858631+0000', 'last_peered': '2024-09-15T21:19:51.858631+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:51.858631+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 5570, 'log_dups_size': 0, 'ondisk_log_size': 5570, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22814720, 'num_objects': 5570, 'num_object_clones': 0, 'num_object_copies': 11140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5570, 'num_write_kb': 22280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2203}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5726", 'reported_seq': 10262, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:53.878062+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:53.878062+0000', 'last_peered': '2024-09-15T21:19:53.878062+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:53.878062+0000', 'last_undegraded': '2024-09-15T21:19:53.878062+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 5726, 'log_dups_size': 0, 'ondisk_log_size': 5726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23437312, 'num_objects': 5722, 'num_object_clones': 0, 'num_object_copies': 11444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1183, 'num_objects_unfound': 0, 'num_objects_dirty': 5722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4449, 'num_write_kb': 17796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2210, 'num_bytes_recovered': 9048064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1183}, {'shards': '2', 'objects': 4539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5641", 'reported_seq': 5766, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.785533+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:51.785533+0000', 'last_peered': '2024-09-15T21:19:51.785533+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:51.785533+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 5641, 'log_dups_size': 0, 'ondisk_log_size': 5641, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23085056, 'num_objects': 5636, 'num_object_clones': 0, 'num_object_copies': 11272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3066, 'num_write_kb': 12264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2265}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5796", 'reported_seq': 5909, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.856691+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:51.856691+0000', 'last_peered': '2024-09-15T21:19:51.856691+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:51.856691+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 5796, 'log_dups_size': 0, 'ondisk_log_size': 5796, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23715840, 'num_objects': 5790, 'num_object_clones': 0, 'num_object_copies': 11580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3432, 'num_write_kb': 13728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2319}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5641", 'reported_seq': 5760, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:53.820391+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:53.820391+0000', 'last_peered': '2024-09-15T21:19:53.820391+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:53.820391+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 5641, 'log_dups_size': 0, 'ondisk_log_size': 5641, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23076864, 'num_objects': 5634, 'num_object_clones': 0, 'num_object_copies': 11268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4227, 'num_write_kb': 16908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 2296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5574", 'reported_seq': 5692, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.783711+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:51.783711+0000', 'last_peered': '2024-09-15T21:19:51.783711+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:51.783711+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 5574, 'log_dups_size': 0, 'ondisk_log_size': 5574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22798336, 'num_objects': 5566, 'num_object_clones': 0, 'num_object_copies': 11132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3100, 'num_write_kb': 12400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2225}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:19:55.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5624", 'reported_seq': 5729, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:53.837498+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:53.837498+0000', 'last_peered': '2024-09-15T21:19:53.837498+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:53.837498+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 5624, 'log_dups_size': 0, 'ondisk_log_size': 5624, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23015424, 'num_objects': 5619, 'num_object_clones': 0, 'num_object_copies': 11238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4175, 'num_write_kb': 16700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 2353}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5693", 'reported_seq': 5803, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:53.831015+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:53.831015+0000', 'last_peered': '2024-09-15T21:19:53.831015+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:53.831015+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 5693, 'log_dups_size': 0, 'ondisk_log_size': 5693, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23293952, 'num_objects': 5687, 'num_object_clones': 0, 'num_object_copies': 11374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4092, 'num_write_kb': 16368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 2337}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5690", 'reported_seq': 5813, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.812247+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:51.812247+0000', 'last_peered': '2024-09-15T21:19:51.812247+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:51.812247+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 5690, 'log_dups_size': 0, 'ondisk_log_size': 5690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23298048, 'num_objects': 5688, 'num_object_clones': 0, 'num_object_copies': 11376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2997, 'num_write_kb': 11988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2255}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5639", 'reported_seq': 5764, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.874977+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:51.874977+0000', 'last_peered': '2024-09-15T21:19:51.874977+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:51.874977+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 5639, 'log_dups_size': 0, 'ondisk_log_size': 5639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23064576, 'num_objects': 5631, 'num_object_clones': 0, 'num_object_copies': 11262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3224, 'num_write_kb': 12896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2252}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5653", 'reported_seq': 5772, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.818724+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:51.818724+0000', 'last_peered': '2024-09-15T21:19:51.818724+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:51.818724+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 5653, 'log_dups_size': 0, 'ondisk_log_size': 5653, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23138304, 'num_objects': 5649, 'num_object_clones': 0, 'num_object_copies': 11298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3300, 'num_write_kb': 13200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2284}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:19:55.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5704", 'reported_seq': 5828, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:51.741818+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:51.741818+0000', 'last_peered': '2024-09-15T21:19:51.741818+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:51.741818+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 5704, 'log_dups_size': 0, 'ondisk_log_size': 5704, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23315049, 'num_objects': 5693, 'num_object_clones': 0, 'num_object_copies': 11386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3364, 'num_write_kb': 13456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2223}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:19:55.619 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:19:56.493 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 123 20 92834 92814 2.94702 2.17969 0.0216313 0.0264977 2024-09-15T21:19:56.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:56.759 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/181322 objects degraded (20.501%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90661,"data_bytes":371335804,"bytes_used":1221373952,"bytes_avail":288688918528,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":181322,"degraded_ratio":0.20500545990006727,"misplaced_objects":1183,"misplaced_total":181322,"misplaced_ratio":0.0065243048278752717,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105733,"recovering_keys_per_sec":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":1960842,"read_op_per_sec":0,"write_op_per_sec":478},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:19:44.337807+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:19:56.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:19:56.826 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:57.441 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:19:57.442 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:19:57.470 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-15T21:19:56.346802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371335804,"num_objects":90661,"num_object_clones":0,"num_object_copies":181322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62156,"num_write_kb":248618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":9048083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90743,"ondisk_log_size":90743,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1192748,"kb_used_data":732604,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281922772,"statfs":{"total":289910292480,"available":288688918528,"internally_reserved":0,"allocated":750186496,"data_stored":749347087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":471125002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,5,5,16,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23547904,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-310,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5749,"num_write_kb":22996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009075"},"pg_stats":[{"pgid":"2.f","version":"44'5570","reported_seq":5642,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.858631+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:51.858631+0000","last_peered":"2024-09-15T21:19:51.858631+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:51.858631+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5570,"num_write_kb":22280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2203},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5737","reported_seq":5828,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865979+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:53.865979+0000","last_peered":"2024-09-15T21:19:53.865979+0000","last_clean":"2024-09-15T21:19:53.865979+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:53.865979+0000","last_undegraded":"2024-09-15T21:19:53.865979+0000","last_fullsized":"2024-09-15T21:19:53.865979+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5737,"log_dups_size":0,"ondisk_log_size":5737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23474176,"num_objects":5731,"num_object_clones":0,"num_object_copies":11462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":17852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5726","reported_seq":10262,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:53.878062+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:53.878062+0000","last_peered":"2024-09-15T21:19:53.878062+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:53.878062+0000","last_undegraded":"2024-09-15T21:19:53.878062+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4449,"num_write_kb":17796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1183},{"shards":"2","objects":4539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5767","reported_seq":5832,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:53.865474+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:53.865474+0000","last_peered":"2024-09-15T21:19:53.865474+0000","last_clean":"2024-09-15T21:19:53.865474+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:53.865474+0000","last_undegraded":"2024-09-15T21:19:53.865474+0000","last_fullsized":"2024-09-15T21:19:53.865474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5767,"log_dups_size":0,"ondisk_log_size":5767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5767,"num_write_kb":23068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5693","reported_seq":5807,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.779592+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:51.779592+0000","last_peered":"2024-09-15T21:19:51.779592+0000","last_clean":"2024-09-15T21:19:51.779592+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:51.779592+0000","last_undegraded":"2024-09-15T21:19:51.779592+0000","last_fullsized":"2024-09-15T21:19:51.779592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23289856,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3551,"num_write_kb":14204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5641","reported_seq":5766,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.785533+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:51.785533+0000","last_peered":"2024-09-15T21:19:51.785533+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:51.785533+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3066,"num_write_kb":12264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5796","reported_seq":5909,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.856691+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:51.856691+0000","last_peered":"2024-09-15T21:19:51.856691+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:51.856691+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3432,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5592","reported_seq":5707,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:51.858777+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:51.858777+0000","last_peered":"2024-09-15T21:19:51.858777+0000","last_clean":"2024-09-15T21:19:51.858777+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:51.858777+0000","last_undegraded":"2024-09-15T21:19:51.858777+0000","last_fullsized":"2024-09-15T21:19:51.858777+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22896640,"num_objects":5590,"num_object_clones":0,"num_object_copies":11180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5641","reported_seq":5760,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.820391+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:53.820391+0000","last_peered":"2024-09-15T21:19:53.820391+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:53.820391+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5641,"log_dups_size":0,"ondisk_log_size":5641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23076864,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5574","reported_seq":5692,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.783711+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:51.783711+0000","last_peered":"2024-09-15T21:19:51.783711+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:51.783711+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2225},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5624","reported_seq":5729,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.837498+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:53.837498+0000","last_peered":"2024-09-15T21:19:53.837498+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:53.837498+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5624,"log_dups_size":0,"ondisk_log_size":5624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23015424,"num_objects":5619,"num_object_clones":0,"num_object_copies":11238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4175,"num_write_kb":16700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2353}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5693","reported_seq":5803,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:53.831015+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:53.831015+0000","last_peered":"2024-09-15T21:19:53.831015+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:53.831015+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4092,"num_write_kb":16368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5690","reported_seq":5813,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.812247+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:51.812247+0000","last_peered":"2024-09-15T21:19:51.812247+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:51.812247+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5690,"log_dups_size":0,"ondisk_log_size":5690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2997,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2255},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5639","reported_seq":5764,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.874977+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:51.874977+0000","last_peered":"2024-09-15T21:19:51.874977+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:51.874977+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5639,"log_dups_size":0,"ondisk_log_size":5639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23064576,"num_objects":5631,"num_object_clones":0,"num_object_copies":11262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":12896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2252},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5653","reported_seq":5772,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.818724+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:51.818724+0000","last_peered":"2024-09-15T21:19:51.818724+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:51.818724+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5653,"log_dups_size":0,"ondisk_log_size":5653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3300,"num_write_kb":13200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2284},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5704","reported_seq":5828,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:51.741818+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:51.741818+0000","last_peered":"2024-09-15T21:19:51.741818+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:51.741818+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5704,"log_dups_size":0,"ondisk_log_size":5704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23315049,"num_objects":5693,"num_object_clones":0,"num_object_copies":11386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3364,"num_write_kb":13456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2223},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371335785,"num_objects":90659,"num_object_clones":0,"num_object_copies":181318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":1183,"num_objects_unfound":0,"num_objects_dirty":90659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62154,"num_write_kb":248616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":9048064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750120960,"data_stored":750037464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90740,"ondisk_log_size":90740,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953504,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430816,"kb_used_data":290832,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93941024,"statfs":{"total":96636764160,"available":96195608576,"internally_reserved":0,"allocated":297811968,"data_stored":297556518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149731,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500864,"kb_used_data":324848,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93870976,"statfs":{"total":96636764160,"available":96123879424,"internally_reserved":0,"allocated":332644352,"data_stored":332388902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,3,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626449,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261068,"kb_used_data":116924,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94110772,"statfs":{"total":96636764160,"available":96369430528,"internally_reserved":0,"allocated":119730176,"data_stored":119401667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332623872,"data_stored":332620393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":119410688,"data_stored":119334150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298086400,"data_stored":298082921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:57.493 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 124 20 93292 93272 2.93768 1.78906 0.0278563 0.0265728 2024-09-15T21:19:58.495 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 125 20 93813 93793 2.9304 2.03516 0.0560355 0.0266516 2024-09-15T21:19:59.496 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 126 20 94403 94383 2.92544 2.30469 0.0195018 0.0267 2024-09-15T21:20:00.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:00.498 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 127 20 94979 94959 2.92006 2.25 0.0214857 0.0267469 2024-09-15T21:20:01.262 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:01.263 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:20:00.347693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382579324,"num_objects":93406,"num_object_clones":0,"num_object_copies":186812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":93406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64901,"num_write_kb":259598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2442,"num_bytes_recovered":9994259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93488,"ondisk_log_size":93488,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1232524,"kb_used_data":755740,"kb_used_omap":12,"kb_used_meta":476723,"kb_avail":281882996,"statfs":{"total":289910292480,"available":288648187904,"internally_reserved":0,"allocated":773877760,"data_stored":773038351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":488164362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,7,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-541,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6661,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":2215936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009022"},"pg_stats":[{"pgid":"2.f","version":"44'5766","reported_seq":5838,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.754788+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:56.754788+0000","last_peered":"2024-09-15T21:19:56.754788+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:56.754788+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23617536,"num_objects":5766,"num_object_clones":0,"num_object_copies":11532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2399},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5889","reported_seq":5981,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.899538+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:58.899538+0000","last_peered":"2024-09-15T21:19:58.899538+0000","last_clean":"2024-09-15T21:19:58.899538+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:58.899538+0000","last_undegraded":"2024-09-15T21:19:58.899538+0000","last_fullsized":"2024-09-15T21:19:58.899538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5889,"log_dups_size":0,"ondisk_log_size":5889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5891","reported_seq":10889,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:58.887383+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:58.887383+0000","last_peered":"2024-09-15T21:19:58.887383+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:58.887383+0000","last_undegraded":"2024-09-15T21:19:58.887383+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4614,"num_write_kb":18456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":4935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5935","reported_seq":6000,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.864219+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:58.864219+0000","last_peered":"2024-09-15T21:19:58.864219+0000","last_clean":"2024-09-15T21:19:58.864219+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:58.864219+0000","last_undegraded":"2024-09-15T21:19:58.864219+0000","last_fullsized":"2024-09-15T21:19:58.864219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5935,"log_dups_size":0,"ondisk_log_size":5935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5935,"num_write_kb":23740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5865","reported_seq":5978,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:56.862534+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:56.862534+0000","last_peered":"2024-09-15T21:19:56.862534+0000","last_clean":"2024-09-15T21:19:56.862534+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:56.862534+0000","last_undegraded":"2024-09-15T21:19:56.862534+0000","last_fullsized":"2024-09-15T21:19:56.862534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5865,"log_dups_size":0,"ondisk_log_size":5865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3723,"num_write_kb":14892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5821","reported_seq":5946,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.853059+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:56.853059+0000","last_peered":"2024-09-15T21:19:56.853059+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:56.853059+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23822336,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3246,"num_write_kb":12984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5996","reported_seq":6114,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.887864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:56.887864+0000","last_peered":"2024-09-15T21:19:56.887864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:56.887864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5996,"log_dups_size":0,"ondisk_log_size":5996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24535040,"num_objects":5990,"num_object_clones":0,"num_object_copies":11980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3632,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2519},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5769","reported_seq":5884,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:56.840040+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:56.840040+0000","last_peered":"2024-09-15T21:19:56.840040+0000","last_clean":"2024-09-15T21:19:56.840040+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:56.840040+0000","last_undegraded":"2024-09-15T21:19:56.840040+0000","last_fullsized":"2024-09-15T21:19:56.840040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5769,"log_dups_size":0,"ondisk_log_size":5769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5801","reported_seq":5921,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.821756+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:58.821756+0000","last_peered":"2024-09-15T21:19:58.821756+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:58.821756+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23732224,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5708","reported_seq":5826,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.764430+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:56.764430+0000","last_peered":"2024-09-15T21:19:56.764430+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:56.764430+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3234,"num_write_kb":12936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5778","reported_seq":5883,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.867926+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:58.867926+0000","last_peered":"2024-09-15T21:19:58.867926+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:58.867926+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23646208,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4329,"num_write_kb":17316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2507}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5872","reported_seq":5982,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.828769+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:58.828769+0000","last_peered":"2024-09-15T21:19:58.828769+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:58.828769+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5872,"log_dups_size":0,"ondisk_log_size":5872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5881","reported_seq":6003,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.886196+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:56.886196+0000","last_peered":"2024-09-15T21:19:56.886196+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:56.886196+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24080384,"num_objects":5879,"num_object_clones":0,"num_object_copies":11758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5800","reported_seq":5925,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.846758+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:56.846758+0000","last_peered":"2024-09-15T21:19:56.846758+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:56.846758+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2413},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5815","reported_seq":5934,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.840665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:56.840665+0000","last_peered":"2024-09-15T21:19:56.840665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:56.840665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5898","reported_seq":6022,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.811775+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:56.811775+0000","last_peered":"2024-09-15T21:19:56.811775+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:56.811775+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5898,"log_dups_size":0,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109673,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3558,"num_write_kb":14232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2417},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":382579305,"num_objects":93404,"num_object_clones":0,"num_object_copies":186808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":93404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64899,"num_write_kb":259596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773734400,"data_stored":773650904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93485,"ondisk_log_size":93485,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953505,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444532,"kb_used_data":296228,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93927308,"statfs":{"total":96636764160,"available":96181563392,"internally_reserved":0,"allocated":303337472,"data_stored":303082022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149732,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513448,"kb_used_data":333272,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93858392,"statfs":{"total":96636764160,"available":96110993408,"internally_reserved":0,"allocated":341270528,"data_stored":341015078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":184479409},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626450,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274544,"kb_used_data":126240,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94097296,"statfs":{"total":96636764160,"available":96355631104,"internally_reserved":0,"allocated":129269760,"data_stored":128941251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":341299200,"data_stored":341295721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":128921600,"data_stored":128845062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303513600,"data_stored":303510121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:01.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:01.498 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 128 20 95530 95510 2.91406 2.15234 0.0132506 0.0267975 2024-09-15T21:20:02.291 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:02.292 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:02.345 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:20:00.347693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382579324,"num_objects":93406,"num_object_clones":0,"num_object_copies":186812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":93406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64901,"num_write_kb":259598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2442,"num_bytes_recovered":9994259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93488,"ondisk_log_size":93488,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1232524,"kb_used_data":755740,"kb_used_omap":12,"kb_used_meta":476723,"kb_avail":281882996,"statfs":{"total":289910292480,"available":288648187904,"internally_reserved":0,"allocated":773877760,"data_stored":773038351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":488164362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,7,8,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-541,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6661,"num_write_kb":26644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":2215936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009022"},"pg_stats":[{"pgid":"2.f","version":"44'5766","reported_seq":5838,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.754788+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:19:56.754788+0000","last_peered":"2024-09-15T21:19:56.754788+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:19:56.754788+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5766,"log_dups_size":0,"ondisk_log_size":5766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23617536,"num_objects":5766,"num_object_clones":0,"num_object_copies":11532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2399},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5889","reported_seq":5981,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.899538+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:58.899538+0000","last_peered":"2024-09-15T21:19:58.899538+0000","last_clean":"2024-09-15T21:19:58.899538+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:58.899538+0000","last_undegraded":"2024-09-15T21:19:58.899538+0000","last_fullsized":"2024-09-15T21:19:58.899538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5889,"log_dups_size":0,"ondisk_log_size":5889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5891","reported_seq":10889,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:58.887383+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:58.887383+0000","last_peered":"2024-09-15T21:19:58.887383+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:58.887383+0000","last_undegraded":"2024-09-15T21:19:58.887383+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4614,"num_write_kb":18456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":4935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5935","reported_seq":6000,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.864219+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:58.864219+0000","last_peered":"2024-09-15T21:19:58.864219+0000","last_clean":"2024-09-15T21:19:58.864219+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:58.864219+0000","last_undegraded":"2024-09-15T21:19:58.864219+0000","last_fullsized":"2024-09-15T21:19:58.864219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5935,"log_dups_size":0,"ondisk_log_size":5935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5935,"num_write_kb":23740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'5865","reported_seq":5978,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:56.862534+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:19:56.862534+0000","last_peered":"2024-09-15T21:19:56.862534+0000","last_clean":"2024-09-15T21:19:56.862534+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:19:56.862534+0000","last_undegraded":"2024-09-15T21:19:56.862534+0000","last_fullsized":"2024-09-15T21:19:56.862534+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":5865,"log_dups_size":0,"ondisk_log_size":5865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23994368,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3723,"num_write_kb":14892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5821","reported_seq":5946,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.853059+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:19:56.853059+0000","last_peered":"2024-09-15T21:19:56.853059+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:19:56.853059+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5821,"log_dups_size":0,"ondisk_log_size":5821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23822336,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3246,"num_write_kb":12984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2445},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'5996","reported_seq":6114,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.887864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:19:56.887864+0000","last_peered":"2024-09-15T21:19:56.887864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:19:56.887864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":5996,"log_dups_size":0,"ondisk_log_size":5996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24535040,"num_objects":5990,"num_object_clones":0,"num_object_copies":11980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3632,"num_write_kb":14528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2519},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5769","reported_seq":5884,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:56.840040+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:19:56.840040+0000","last_peered":"2024-09-15T21:19:56.840040+0000","last_clean":"2024-09-15T21:19:56.840040+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:19:56.840040+0000","last_undegraded":"2024-09-15T21:19:56.840040+0000","last_fullsized":"2024-09-15T21:19:56.840040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5769,"log_dups_size":0,"ondisk_log_size":5769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5801","reported_seq":5921,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.821756+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:58.821756+0000","last_peered":"2024-09-15T21:19:58.821756+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:58.821756+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23732224,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5708","reported_seq":5826,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.764430+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:19:56.764430+0000","last_peered":"2024-09-15T21:19:56.764430+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:19:56.764430+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3234,"num_write_kb":12936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5778","reported_seq":5883,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.867926+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:58.867926+0000","last_peered":"2024-09-15T21:19:58.867926+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:58.867926+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23646208,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4329,"num_write_kb":17316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2507}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5872","reported_seq":5982,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.828769+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:58.828769+0000","last_peered":"2024-09-15T21:19:58.828769+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:58.828769+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5872,"log_dups_size":0,"ondisk_log_size":5872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'5881","reported_seq":6003,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.886196+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:19:56.886196+0000","last_peered":"2024-09-15T21:19:56.886196+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:19:56.886196+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24080384,"num_objects":5879,"num_object_clones":0,"num_object_copies":11758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5800","reported_seq":5925,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.846758+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:19:56.846758+0000","last_peered":"2024-09-15T21:19:56.846758+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:19:56.846758+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3385,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2413},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5815","reported_seq":5934,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.840665+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:19:56.840665+0000","last_peered":"2024-09-15T21:19:56.840665+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:19:56.840665+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'5898","reported_seq":6022,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:56.811775+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:19:56.811775+0000","last_peered":"2024-09-15T21:19:56.811775+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:19:56.811775+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":5898,"log_dups_size":0,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109673,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3558,"num_write_kb":14232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2417},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":382579305,"num_objects":93404,"num_object_clones":0,"num_object_copies":186808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":93404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64899,"num_write_kb":259596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773734400,"data_stored":773650904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93485,"ondisk_log_size":93485,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953505,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444532,"kb_used_data":296228,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93927308,"statfs":{"total":96636764160,"available":96181563392,"internally_reserved":0,"allocated":303337472,"data_stored":303082022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149732,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513448,"kb_used_data":333272,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93858392,"statfs":{"total":96636764160,"available":96110993408,"internally_reserved":0,"allocated":341270528,"data_stored":341015078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":184479409},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626450,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274544,"kb_used_data":126240,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94097296,"statfs":{"total":96636764160,"available":96355631104,"internally_reserved":0,"allocated":129269760,"data_stored":128941251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":151842476},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":341299200,"data_stored":341295721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":128921600,"data_stored":128845062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303513600,"data_stored":303510121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:02.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:02.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'5766", 'reported_seq': 5838, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.754788+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:19:56.754788+0000', 'last_peered': '2024-09-15T21:19:56.754788+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:19:56.754788+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 5766, 'log_dups_size': 0, 'ondisk_log_size': 5766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23617536, 'num_objects': 5766, 'num_object_clones': 0, 'num_object_copies': 11532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5766, 'num_write_kb': 23064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2399}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'5891", 'reported_seq': 10889, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:19:58.887383+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:19:58.887383+0000', 'last_peered': '2024-09-15T21:19:58.887383+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:19:58.887383+0000', 'last_undegraded': '2024-09-15T21:19:58.887383+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 5891, 'log_dups_size': 0, 'ondisk_log_size': 5891, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113152, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 952, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4614, 'num_write_kb': 18456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2441, 'num_bytes_recovered': 9994240, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 952}, {'shards': '2', 'objects': 4935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'5821", 'reported_seq': 5946, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.853059+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:19:56.853059+0000', 'last_peered': '2024-09-15T21:19:56.853059+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:19:56.853059+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 5821, 'log_dups_size': 0, 'ondisk_log_size': 5821, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23822336, 'num_objects': 5816, 'num_object_clones': 0, 'num_object_copies': 11632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3246, 'num_write_kb': 12984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2445}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'5996", 'reported_seq': 6114, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.887864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:19:56.887864+0000', 'last_peered': '2024-09-15T21:19:56.887864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:19:56.887864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 5996, 'log_dups_size': 0, 'ondisk_log_size': 5996, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24535040, 'num_objects': 5990, 'num_object_clones': 0, 'num_object_copies': 11980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5990, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3632, 'num_write_kb': 14528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2519}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5801", 'reported_seq': 5921, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:58.821756+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:19:58.821756+0000', 'last_peered': '2024-09-15T21:19:58.821756+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:19:58.821756+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 5801, 'log_dups_size': 0, 'ondisk_log_size': 5801, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23732224, 'num_objects': 5794, 'num_object_clones': 0, 'num_object_copies': 11588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4387, 'num_write_kb': 17548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 2456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:02.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'5708", 'reported_seq': 5826, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.764430+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:19:56.764430+0000', 'last_peered': '2024-09-15T21:19:56.764430+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:19:56.764430+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 5708, 'log_dups_size': 0, 'ondisk_log_size': 5708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23347200, 'num_objects': 5700, 'num_object_clones': 0, 'num_object_copies': 11400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3234, 'num_write_kb': 12936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2359}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5778", 'reported_seq': 5883, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:58.867926+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:19:58.867926+0000', 'last_peered': '2024-09-15T21:19:58.867926+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:19:58.867926+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 5778, 'log_dups_size': 0, 'ondisk_log_size': 5778, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23646208, 'num_objects': 5773, 'num_object_clones': 0, 'num_object_copies': 11546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4329, 'num_write_kb': 17316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 2507}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'5872", 'reported_seq': 5982, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:58.828769+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:19:58.828769+0000', 'last_peered': '2024-09-15T21:19:58.828769+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:19:58.828769+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 5872, 'log_dups_size': 0, 'ondisk_log_size': 5872, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24027136, 'num_objects': 5866, 'num_object_clones': 0, 'num_object_copies': 11732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4271, 'num_write_kb': 17084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'5881", 'reported_seq': 6003, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.886196+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:19:56.886196+0000', 'last_peered': '2024-09-15T21:19:56.886196+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:19:56.886196+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 5881, 'log_dups_size': 0, 'ondisk_log_size': 5881, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24080384, 'num_objects': 5879, 'num_object_clones': 0, 'num_object_copies': 11758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3188, 'num_write_kb': 12752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2446}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'5800", 'reported_seq': 5925, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.846758+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:19:56.846758+0000', 'last_peered': '2024-09-15T21:19:56.846758+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:19:56.846758+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 5800, 'log_dups_size': 0, 'ondisk_log_size': 5800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23724032, 'num_objects': 5792, 'num_object_clones': 0, 'num_object_copies': 11584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3385, 'num_write_kb': 13540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2413}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:02.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'5815", 'reported_seq': 5934, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.840665+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:19:56.840665+0000', 'last_peered': '2024-09-15T21:19:56.840665+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:19:56.840665+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 5815, 'log_dups_size': 0, 'ondisk_log_size': 5815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23801856, 'num_objects': 5811, 'num_object_clones': 0, 'num_object_copies': 11622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3462, 'num_write_kb': 13848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2446}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:02.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'5898", 'reported_seq': 6022, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:19:56.811775+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:19:56.811775+0000', 'last_peered': '2024-09-15T21:19:56.811775+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:19:56.811775+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 5898, 'log_dups_size': 0, 'ondisk_log_size': 5898, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24109673, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3558, 'num_write_kb': 14232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2417}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:02.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:02.498 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 129 20 96092 96072 2.90849 2.19531 0.0288934 0.0268514 2024-09-15T21:20:03.028 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:03.028 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/186812 objects degraded (19.898%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":216,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93406,"data_bytes":382579324,"bytes_used":1262104576,"bytes_avail":288648187904,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":186812,"degraded_ratio":0.19898079352504122,"misplaced_objects":952,"misplaced_total":186812,"misplaced_ratio":0.0050960323747939107,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":184522,"recovering_keys_per_sec":0,"num_objects_recovered":541,"num_bytes_recovered":2215936,"num_keys_recovered":0,"write_bytes_sec":2271913,"read_op_per_sec":0,"write_op_per_sec":554},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:19:44.337807+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:20:03.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:03.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:03.498 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 130 20 96550 96530 2.89988 1.78906 0.0128556 0.0269331 2024-09-15T21:20:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:03.792 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:03.808 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-15T21:20:02.348166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389608060,"num_objects":95122,"num_object_clones":0,"num_object_copies":190244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":95122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66617,"num_write_kb":266462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2442,"num_bytes_recovered":9994259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95204,"ondisk_log_size":95204,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1249592,"kb_used_data":772808,"kb_used_omap":12,"kb_used_meta":476723,"kb_avail":281865928,"statfs":{"total":289910292480,"available":288630710272,"internally_reserved":0,"allocated":791355392,"data_stored":790515983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":488164362},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1,3,0,20],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-377,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008997"},"pg_stats":[{"pgid":"2.f","version":"44'5942","reported_seq":6014,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.798574+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:01.798574+0000","last_peered":"2024-09-15T21:20:01.798574+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:01.798574+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24338432,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'5889","reported_seq":5981,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.899538+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:19:58.899538+0000","last_peered":"2024-09-15T21:19:58.899538+0000","last_clean":"2024-09-15T21:19:58.899538+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:19:58.899538+0000","last_undegraded":"2024-09-15T21:19:58.899538+0000","last_fullsized":"2024-09-15T21:19:58.899538+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":5889,"log_dups_size":0,"ondisk_log_size":5889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24096768,"num_objects":5883,"num_object_clones":0,"num_object_copies":11766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'5891","reported_seq":10889,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:19:58.887383+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:19:58.887383+0000","last_peered":"2024-09-15T21:19:58.887383+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:19:58.887383+0000","last_undegraded":"2024-09-15T21:19:58.887383+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":5891,"log_dups_size":0,"ondisk_log_size":5891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4614,"num_write_kb":18456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":4935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'5935","reported_seq":6000,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:19:58.864219+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:19:58.864219+0000","last_peered":"2024-09-15T21:19:58.864219+0000","last_clean":"2024-09-15T21:19:58.864219+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:19:58.864219+0000","last_undegraded":"2024-09-15T21:19:58.864219+0000","last_fullsized":"2024-09-15T21:19:58.864219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5935,"log_dups_size":0,"ondisk_log_size":5935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5935,"num_write_kb":23740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6037","reported_seq":6150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.922208+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:01.922208+0000","last_peered":"2024-09-15T21:20:01.922208+0000","last_clean":"2024-09-15T21:20:01.922208+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:01.922208+0000","last_undegraded":"2024-09-15T21:20:01.922208+0000","last_fullsized":"2024-09-15T21:20:01.922208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6037,"log_dups_size":0,"ondisk_log_size":6037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5996","reported_seq":6120,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.891015+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:01.891015+0000","last_peered":"2024-09-15T21:20:01.891015+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:01.891015+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5996,"log_dups_size":0,"ondisk_log_size":5996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24539136,"num_objects":5991,"num_object_clones":0,"num_object_copies":11982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3421,"num_write_kb":13684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2620},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6160","reported_seq":6278,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.882425+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:01.882425+0000","last_peered":"2024-09-15T21:20:01.882425+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:01.882425+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6160,"log_dups_size":0,"ondisk_log_size":6160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3796,"num_write_kb":15184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2683},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5968","reported_seq":6083,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.904642+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:01.904642+0000","last_peered":"2024-09-15T21:20:01.904642+0000","last_clean":"2024-09-15T21:20:01.904642+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:01.904642+0000","last_undegraded":"2024-09-15T21:20:01.904642+0000","last_fullsized":"2024-09-15T21:20:01.904642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5968,"log_dups_size":0,"ondisk_log_size":5968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3753,"num_write_kb":15012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5801","reported_seq":5921,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.821756+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:19:58.821756+0000","last_peered":"2024-09-15T21:19:58.821756+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:19:58.821756+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5801,"log_dups_size":0,"ondisk_log_size":5801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23732224,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4387,"num_write_kb":17548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5897","reported_seq":6014,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.897632+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:01.897632+0000","last_peered":"2024-09-15T21:20:01.897632+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:01.897632+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5897,"log_dups_size":0,"ondisk_log_size":5897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24121344,"num_objects":5889,"num_object_clones":0,"num_object_copies":11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2548},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5778","reported_seq":5883,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.867926+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:19:58.867926+0000","last_peered":"2024-09-15T21:19:58.867926+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:19:58.867926+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23646208,"num_objects":5773,"num_object_clones":0,"num_object_copies":11546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4329,"num_write_kb":17316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2507}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'5872","reported_seq":5982,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:19:58.828769+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:19:58.828769+0000","last_peered":"2024-09-15T21:19:58.828769+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:19:58.828769+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":5872,"log_dups_size":0,"ondisk_log_size":5872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6051","reported_seq":6174,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.928646+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:01.928646+0000","last_peered":"2024-09-15T21:20:01.928646+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:01.928646+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3358,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5947","reported_seq":6072,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.905799+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:01.905799+0000","last_peered":"2024-09-15T21:20:01.905799+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:01.905799+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24326144,"num_objects":5939,"num_object_clones":0,"num_object_copies":11878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":14128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2560},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5970","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.846714+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:01.846714+0000","last_peered":"2024-09-15T21:20:01.846714+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:01.846714+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3617,"num_write_kb":14468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2601},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6067","reported_seq":6191,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.875144+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:01.875144+0000","last_peered":"2024-09-15T21:20:01.875144+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:01.875144+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24801897,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3727,"num_write_kb":14908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389608041,"num_objects":95120,"num_object_clones":0,"num_object_copies":190240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":952,"num_objects_unfound":0,"num_objects_dirty":95120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66615,"num_write_kb":266460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2441,"num_bytes_recovered":9994240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791298048,"data_stored":791214552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95201,"ondisk_log_size":95201,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953505,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444532,"kb_used_data":296228,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93927308,"statfs":{"total":96636764160,"available":96181563392,"internally_reserved":0,"allocated":303337472,"data_stored":303082022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":151842477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149733,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529988,"kb_used_data":341492,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93841852,"statfs":{"total":96636764160,"available":96094056448,"internally_reserved":0,"allocated":349687808,"data_stored":349432358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":192999089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,20],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":275072,"kb_used_data":135088,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94096768,"statfs":{"total":96636764160,"available":96355090432,"internally_reserved":0,"allocated":138330112,"data_stored":138001603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349687808,"data_stored":349684329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138096640,"data_stored":138020102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303513600,"data_stored":303510121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:04.498 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 131 20 96898 96878 2.88812 1.35938 0.034028 0.0270428 2024-09-15T21:20:05.500 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 132 20 97473 97453 2.88322 2.24609 0.0212401 0.0270874 2024-09-15T21:20:06.500 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 133 20 97959 97939 2.87582 1.89844 0.00949268 0.0271508 2024-09-15T21:20:06.811 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:07.502 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 134 20 98521 98501 2.8707 2.19531 0.0163226 0.0272051 2024-09-15T21:20:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:07.636 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:07.711 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-15T21:20:06.349164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393659004,"num_objects":96111,"num_object_clones":0,"num_object_copies":192222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":96111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67606,"num_write_kb":270418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2683,"num_bytes_recovered":10981395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96193,"ondisk_log_size":96193,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1259228,"kb_used_data":778284,"kb_used_omap":12,"kb_used_meta":480883,"kb_avail":281856292,"statfs":{"total":289910292480,"available":288620843008,"internally_reserved":0,"allocated":796962816,"data_stored":796123407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":492424202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,0,3,1,20],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22323200,"num_objects":5450,"num_object_clones":0,"num_object_copies":10900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-472,"num_objects_unfound":0,"num_objects_dirty":5450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5450,"num_write_kb":21800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":1933312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002884"},"pg_stats":[{"pgid":"2.f","version":"44'5942","reported_seq":6014,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.798574+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:01.798574+0000","last_peered":"2024-09-15T21:20:01.798574+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:01.798574+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24338432,"num_objects":5942,"num_object_clones":0,"num_object_copies":11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6056","reported_seq":6148,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.700328+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:03.700328+0000","last_peered":"2024-09-15T21:20:03.700328+0000","last_clean":"2024-09-15T21:20:03.700328+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:03.700328+0000","last_undegraded":"2024-09-15T21:20:03.700328+0000","last_fullsized":"2024-09-15T21:20:03.700328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6056,"log_dups_size":0,"ondisk_log_size":6056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4782,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6049","reported_seq":11530,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:03.900147+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:03.900147+0000","last_peered":"2024-09-15T21:20:03.900147+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:03.900147+0000","last_undegraded":"2024-09-15T21:20:03.900147+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6117","reported_seq":6182,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.706817+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:03.706817+0000","last_peered":"2024-09-15T21:20:03.706817+0000","last_clean":"2024-09-15T21:20:03.706817+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:03.706817+0000","last_undegraded":"2024-09-15T21:20:03.706817+0000","last_fullsized":"2024-09-15T21:20:03.706817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6117,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6037","reported_seq":6150,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.922208+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:01.922208+0000","last_peered":"2024-09-15T21:20:01.922208+0000","last_clean":"2024-09-15T21:20:01.922208+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:01.922208+0000","last_undegraded":"2024-09-15T21:20:01.922208+0000","last_fullsized":"2024-09-15T21:20:01.922208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6037,"log_dups_size":0,"ondisk_log_size":6037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'5996","reported_seq":6120,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.891015+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:01.891015+0000","last_peered":"2024-09-15T21:20:01.891015+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:01.891015+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":5996,"log_dups_size":0,"ondisk_log_size":5996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24539136,"num_objects":5991,"num_object_clones":0,"num_object_copies":11982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3421,"num_write_kb":13684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2620},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6160","reported_seq":6278,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.882425+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:01.882425+0000","last_peered":"2024-09-15T21:20:01.882425+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:01.882425+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6160,"log_dups_size":0,"ondisk_log_size":6160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3796,"num_write_kb":15184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2683},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'5968","reported_seq":6083,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.904642+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:01.904642+0000","last_peered":"2024-09-15T21:20:01.904642+0000","last_clean":"2024-09-15T21:20:01.904642+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:01.904642+0000","last_undegraded":"2024-09-15T21:20:01.904642+0000","last_fullsized":"2024-09-15T21:20:01.904642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":5968,"log_dups_size":0,"ondisk_log_size":5968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3753,"num_write_kb":15012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5960","reported_seq":6080,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.680726+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:03.680726+0000","last_peered":"2024-09-15T21:20:03.680726+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:03.680726+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'5897","reported_seq":6014,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.897632+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:01.897632+0000","last_peered":"2024-09-15T21:20:01.897632+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:01.897632+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":5897,"log_dups_size":0,"ondisk_log_size":5897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24121344,"num_objects":5889,"num_object_clones":0,"num_object_copies":11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3423,"num_write_kb":13692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2548},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5943","reported_seq":6048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.588199+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:03.588199+0000","last_peered":"2024-09-15T21:20:03.588199+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:03.588199+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24322048,"num_objects":5938,"num_object_clones":0,"num_object_copies":11876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4494,"num_write_kb":17976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2672}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6030","reported_seq":6140,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.601559+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:03.601559+0000","last_peered":"2024-09-15T21:20:03.601559+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:03.601559+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674304,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":17716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2674}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6051","reported_seq":6174,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.928646+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:01.928646+0000","last_peered":"2024-09-15T21:20:01.928646+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:01.928646+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6051,"log_dups_size":0,"ondisk_log_size":6051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3358,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2616},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'5947","reported_seq":6072,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.905799+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:01.905799+0000","last_peered":"2024-09-15T21:20:01.905799+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:01.905799+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24326144,"num_objects":5939,"num_object_clones":0,"num_object_copies":11878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3532,"num_write_kb":14128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2560},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'5970","reported_seq":6089,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.846714+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:01.846714+0000","last_peered":"2024-09-15T21:20:01.846714+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:01.846714+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24436736,"num_objects":5966,"num_object_clones":0,"num_object_copies":11932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3617,"num_write_kb":14468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2601},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6067","reported_seq":6191,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:01.875144+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:01.875144+0000","last_peered":"2024-09-15T21:20:01.875144+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:01.875144+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24801897,"num_objects":6056,"num_object_clones":0,"num_object_copies":12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3727,"num_write_kb":14908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393658985,"num_objects":96109,"num_object_clones":0,"num_object_copies":192218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":96109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67604,"num_write_kb":270416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":796753920,"data_stored":796670424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96190,"ondisk_log_size":96190,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454168,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93917672,"statfs":{"total":96636764160,"available":96171696128,"internally_reserved":0,"allocated":308944896,"data_stored":308689446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149733,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529988,"kb_used_data":341492,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93841852,"statfs":{"total":96636764160,"available":96094056448,"internally_reserved":0,"allocated":349687808,"data_stored":349432358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":192999089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,20],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":275072,"kb_used_data":135088,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94096768,"statfs":{"total":96636764160,"available":96355090432,"internally_reserved":0,"allocated":138330112,"data_stored":138001603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":143322796},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349687808,"data_stored":349684329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138096640,"data_stored":138020102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308969472,"data_stored":308965993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:07.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:08.502 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 135 20 99061 99041 2.86506 2.10938 0.0185595 0.0272573 2024-09-15T21:20:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:08.504 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:08.526 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-15T21:20:08.349504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399958652,"num_objects":97649,"num_object_clones":0,"num_object_copies":195298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":97649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69144,"num_write_kb":276570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2683,"num_bytes_recovered":10981395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97731,"ondisk_log_size":97731,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1266544,"kb_used_data":793920,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281848976,"statfs":{"total":289910292480,"available":288613351424,"internally_reserved":0,"allocated":812974080,"data_stored":812134671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":483904522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,11,3,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-472,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":1933312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002702"},"pg_stats":[{"pgid":"2.f","version":"44'6096","reported_seq":6168,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.971333+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:06.971333+0000","last_peered":"2024-09-15T21:20:06.971333+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:06.971333+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6096,"log_dups_size":0,"ondisk_log_size":6096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6096,"num_write_kb":24384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6056","reported_seq":6148,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.700328+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:03.700328+0000","last_peered":"2024-09-15T21:20:03.700328+0000","last_clean":"2024-09-15T21:20:03.700328+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:03.700328+0000","last_undegraded":"2024-09-15T21:20:03.700328+0000","last_fullsized":"2024-09-15T21:20:03.700328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6056,"log_dups_size":0,"ondisk_log_size":6056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4782,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6049","reported_seq":11530,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:03.900147+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:03.900147+0000","last_peered":"2024-09-15T21:20:03.900147+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:03.900147+0000","last_undegraded":"2024-09-15T21:20:03.900147+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6117","reported_seq":6182,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.706817+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:03.706817+0000","last_peered":"2024-09-15T21:20:03.706817+0000","last_clean":"2024-09-15T21:20:03.706817+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:03.706817+0000","last_undegraded":"2024-09-15T21:20:03.706817+0000","last_fullsized":"2024-09-15T21:20:03.706817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6117,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6193","reported_seq":6307,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:06.970453+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:06.970453+0000","last_peered":"2024-09-15T21:20:06.970453+0000","last_clean":"2024-09-15T21:20:06.970453+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:06.970453+0000","last_undegraded":"2024-09-15T21:20:06.970453+0000","last_fullsized":"2024-09-15T21:20:06.970453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6193,"log_dups_size":0,"ondisk_log_size":6193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25337856,"num_objects":6186,"num_object_clones":0,"num_object_copies":12372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4051,"num_write_kb":16204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6143","reported_seq":6268,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.879501+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:06.879501+0000","last_peered":"2024-09-15T21:20:06.879501+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:06.879501+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6143,"log_dups_size":0,"ondisk_log_size":6143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25141248,"num_objects":6138,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3568,"num_write_kb":14272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2767},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6317","reported_seq":6435,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.915392+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:06.915392+0000","last_peered":"2024-09-15T21:20:06.915392+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:06.915392+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6317,"log_dups_size":0,"ondisk_log_size":6317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3953,"num_write_kb":15812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2840},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6119","reported_seq":6234,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:06.930676+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:06.930676+0000","last_peered":"2024-09-15T21:20:06.930676+0000","last_clean":"2024-09-15T21:20:06.930676+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:06.930676+0000","last_undegraded":"2024-09-15T21:20:06.930676+0000","last_fullsized":"2024-09-15T21:20:06.930676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3904,"num_write_kb":15616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5960","reported_seq":6080,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.680726+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:03.680726+0000","last_peered":"2024-09-15T21:20:03.680726+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:03.680726+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6050","reported_seq":6164,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.972377+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:06.972377+0000","last_peered":"2024-09-15T21:20:06.972377+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:06.972377+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24748032,"num_objects":6042,"num_object_clones":0,"num_object_copies":12084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3576,"num_write_kb":14304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2701},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5943","reported_seq":6048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.588199+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:03.588199+0000","last_peered":"2024-09-15T21:20:03.588199+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:03.588199+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24322048,"num_objects":5938,"num_object_clones":0,"num_object_copies":11876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4494,"num_write_kb":17976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2672}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6030","reported_seq":6140,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.601559+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:03.601559+0000","last_peered":"2024-09-15T21:20:03.601559+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:03.601559+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674304,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":17716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2674}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6211","reported_seq":6334,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.819485+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:06.819485+0000","last_peered":"2024-09-15T21:20:06.819485+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:06.819485+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2776},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6084","reported_seq":6208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.939514+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:06.939514+0000","last_peered":"2024-09-15T21:20:06.939514+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:06.939514+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6084,"log_dups_size":0,"ondisk_log_size":6084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3669,"num_write_kb":14676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6119","reported_seq":6238,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.939031+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:06.939031+0000","last_peered":"2024-09-15T21:20:06.939031+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:06.939031+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047040,"num_objects":6115,"num_object_clones":0,"num_object_copies":12230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3766,"num_write_kb":15064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2750},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6241","reported_seq":6365,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.955522+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:06.955522+0000","last_peered":"2024-09-15T21:20:06.955522+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:06.955522+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6241,"log_dups_size":0,"ondisk_log_size":6241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25514601,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3901,"num_write_kb":15604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2760},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399958633,"num_objects":97647,"num_object_clones":0,"num_object_copies":195294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":97647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69142,"num_write_kb":276568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":813051904,"data_stored":812968408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97728,"ondisk_log_size":97728,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454168,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93917672,"statfs":{"total":96636764160,"available":96171696128,"internally_reserved":0,"allocated":308944896,"data_stored":308689446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149734,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524912,"kb_used_data":348896,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93846928,"statfs":{"total":96636764160,"available":96099254272,"internally_reserved":0,"allocated":357269504,"data_stored":357014054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287464,"kb_used_data":143320,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94084376,"statfs":{"total":96636764160,"available":96342401024,"internally_reserved":0,"allocated":146759680,"data_stored":146431171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357335040,"data_stored":357331561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146747392,"data_stored":146670854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308969472,"data_stored":308965993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:08.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:08.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6096", 'reported_seq': 6168, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.971333+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:06.971333+0000', 'last_peered': '2024-09-15T21:20:06.971333+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:06.971333+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 6096, 'log_dups_size': 0, 'ondisk_log_size': 6096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24969216, 'num_objects': 6096, 'num_object_clones': 0, 'num_object_copies': 12192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6096, 'num_write_kb': 24384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2729}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:20:08.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6049", 'reported_seq': 11530, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:03.900147+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:20:03.900147+0000', 'last_peered': '2024-09-15T21:20:03.900147+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:20:03.900147+0000', 'last_undegraded': '2024-09-15T21:20:03.900147+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 6049, 'log_dups_size': 0, 'ondisk_log_size': 6049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24760320, 'num_objects': 6045, 'num_object_clones': 0, 'num_object_copies': 12090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 711, 'num_objects_unfound': 0, 'num_objects_dirty': 6045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4772, 'num_write_kb': 19088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2682, 'num_bytes_recovered': 10981376, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 711}, {'shards': '2', 'objects': 5334}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6143", 'reported_seq': 6268, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.879501+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:06.879501+0000', 'last_peered': '2024-09-15T21:20:06.879501+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:06.879501+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 6143, 'log_dups_size': 0, 'ondisk_log_size': 6143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25141248, 'num_objects': 6138, 'num_object_clones': 0, 'num_object_copies': 12276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3568, 'num_write_kb': 14272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2767}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6317", 'reported_seq': 6435, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.915392+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:06.915392+0000', 'last_peered': '2024-09-15T21:20:06.915392+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:06.915392+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 6317, 'log_dups_size': 0, 'ondisk_log_size': 6317, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25849856, 'num_objects': 6311, 'num_object_clones': 0, 'num_object_copies': 12622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3953, 'num_write_kb': 15812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2840}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'5960", 'reported_seq': 6080, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:03.680726+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:03.680726+0000', 'last_peered': '2024-09-15T21:20:03.680726+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:03.680726+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 5960, 'log_dups_size': 0, 'ondisk_log_size': 5960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24383488, 'num_objects': 5953, 'num_object_clones': 0, 'num_object_copies': 11906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4546, 'num_write_kb': 18184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 2615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6050", 'reported_seq': 6164, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.972377+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:06.972377+0000', 'last_peered': '2024-09-15T21:20:06.972377+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:06.972377+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6050, 'log_dups_size': 0, 'ondisk_log_size': 6050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24748032, 'num_objects': 6042, 'num_object_clones': 0, 'num_object_copies': 12084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3576, 'num_write_kb': 14304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2701}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'5943", 'reported_seq': 6048, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:03.588199+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:03.588199+0000', 'last_peered': '2024-09-15T21:20:03.588199+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:03.588199+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 5943, 'log_dups_size': 0, 'ondisk_log_size': 5943, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24322048, 'num_objects': 5938, 'num_object_clones': 0, 'num_object_copies': 11876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4494, 'num_write_kb': 17976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 2672}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6030", 'reported_seq': 6140, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:03.601559+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:03.601559+0000', 'last_peered': '2024-09-15T21:20:03.601559+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:03.601559+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6030, 'log_dups_size': 0, 'ondisk_log_size': 6030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24674304, 'num_objects': 6024, 'num_object_clones': 0, 'num_object_copies': 12048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4429, 'num_write_kb': 17716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 2674}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6211", 'reported_seq': 6334, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.819485+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:06.819485+0000', 'last_peered': '2024-09-15T21:20:06.819485+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:06.819485+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 6211, 'log_dups_size': 0, 'ondisk_log_size': 6211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25432064, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3518, 'num_write_kb': 14072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2776}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6084", 'reported_seq': 6208, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.939514+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:06.939514+0000', 'last_peered': '2024-09-15T21:20:06.939514+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:06.939514+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6084, 'log_dups_size': 0, 'ondisk_log_size': 6084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24887296, 'num_objects': 6076, 'num_object_clones': 0, 'num_object_copies': 12152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3669, 'num_write_kb': 14676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2697}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6119", 'reported_seq': 6238, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.939031+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:06.939031+0000', 'last_peered': '2024-09-15T21:20:06.939031+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:06.939031+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 6119, 'log_dups_size': 0, 'ondisk_log_size': 6119, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25047040, 'num_objects': 6115, 'num_object_clones': 0, 'num_object_copies': 12230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6115, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3766, 'num_write_kb': 15064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2750}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:08.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6241", 'reported_seq': 6365, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:06.955522+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:20:06.955522+0000', 'last_peered': '2024-09-15T21:20:06.955522+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:06.955522+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 6241, 'log_dups_size': 0, 'ondisk_log_size': 6241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25514601, 'num_objects': 6230, 'num_object_clones': 0, 'num_object_copies': 12460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3901, 'num_write_kb': 15604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2760}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:08.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:09.359 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:09.359 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/195298 objects degraded (19.033%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":222,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97649,"data_bytes":399958652,"bytes_used":1296941056,"bytes_avail":288613351424,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":195298,"degraded_ratio":0.19033477045335842,"misplaced_objects":711,"misplaced_total":195298,"misplaced_ratio":0.0036405902774221959,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":161073,"recovering_keys_per_sec":0,"num_objects_recovered":472,"num_bytes_recovered":1933312,"num_keys_recovered":0,"write_bytes_sec":2384700,"read_op_per_sec":0,"write_op_per_sec":582},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:19:44.337807+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:20:09.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:09.418 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:09.502 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 136 20 99507 99487 2.85681 1.74219 0.0859546 0.0273376 2024-09-15T21:20:10.211 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:10.212 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:10.284 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-15T21:20:08.349504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399958652,"num_objects":97649,"num_object_clones":0,"num_object_copies":195298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":97649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69144,"num_write_kb":276570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2683,"num_bytes_recovered":10981395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97731,"ondisk_log_size":97731,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1266544,"kb_used_data":793920,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281848976,"statfs":{"total":289910292480,"available":288613351424,"internally_reserved":0,"allocated":812974080,"data_stored":812134671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":483904522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,11,3,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-472,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":1933312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002702"},"pg_stats":[{"pgid":"2.f","version":"44'6096","reported_seq":6168,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.971333+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:06.971333+0000","last_peered":"2024-09-15T21:20:06.971333+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:06.971333+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6096,"log_dups_size":0,"ondisk_log_size":6096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6096,"num_write_kb":24384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6056","reported_seq":6148,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.700328+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:03.700328+0000","last_peered":"2024-09-15T21:20:03.700328+0000","last_clean":"2024-09-15T21:20:03.700328+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:03.700328+0000","last_undegraded":"2024-09-15T21:20:03.700328+0000","last_fullsized":"2024-09-15T21:20:03.700328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6056,"log_dups_size":0,"ondisk_log_size":6056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4782,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6049","reported_seq":11530,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:03.900147+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:03.900147+0000","last_peered":"2024-09-15T21:20:03.900147+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:03.900147+0000","last_undegraded":"2024-09-15T21:20:03.900147+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6049,"log_dups_size":0,"ondisk_log_size":6049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24760320,"num_objects":6045,"num_object_clones":0,"num_object_copies":12090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4772,"num_write_kb":19088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6117","reported_seq":6182,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:03.706817+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:03.706817+0000","last_peered":"2024-09-15T21:20:03.706817+0000","last_clean":"2024-09-15T21:20:03.706817+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:03.706817+0000","last_undegraded":"2024-09-15T21:20:03.706817+0000","last_fullsized":"2024-09-15T21:20:03.706817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6117,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6193","reported_seq":6307,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:06.970453+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:06.970453+0000","last_peered":"2024-09-15T21:20:06.970453+0000","last_clean":"2024-09-15T21:20:06.970453+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:06.970453+0000","last_undegraded":"2024-09-15T21:20:06.970453+0000","last_fullsized":"2024-09-15T21:20:06.970453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6193,"log_dups_size":0,"ondisk_log_size":6193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25337856,"num_objects":6186,"num_object_clones":0,"num_object_copies":12372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4051,"num_write_kb":16204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6143","reported_seq":6268,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.879501+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:06.879501+0000","last_peered":"2024-09-15T21:20:06.879501+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:06.879501+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6143,"log_dups_size":0,"ondisk_log_size":6143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25141248,"num_objects":6138,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3568,"num_write_kb":14272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2767},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6317","reported_seq":6435,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.915392+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:06.915392+0000","last_peered":"2024-09-15T21:20:06.915392+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:06.915392+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6317,"log_dups_size":0,"ondisk_log_size":6317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3953,"num_write_kb":15812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2840},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6119","reported_seq":6234,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:06.930676+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:06.930676+0000","last_peered":"2024-09-15T21:20:06.930676+0000","last_clean":"2024-09-15T21:20:06.930676+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:06.930676+0000","last_undegraded":"2024-09-15T21:20:06.930676+0000","last_fullsized":"2024-09-15T21:20:06.930676+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3904,"num_write_kb":15616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'5960","reported_seq":6080,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.680726+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:03.680726+0000","last_peered":"2024-09-15T21:20:03.680726+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:03.680726+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24383488,"num_objects":5953,"num_object_clones":0,"num_object_copies":11906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6050","reported_seq":6164,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.972377+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:06.972377+0000","last_peered":"2024-09-15T21:20:06.972377+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:06.972377+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6050,"log_dups_size":0,"ondisk_log_size":6050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24748032,"num_objects":6042,"num_object_clones":0,"num_object_copies":12084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3576,"num_write_kb":14304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2701},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'5943","reported_seq":6048,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.588199+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:03.588199+0000","last_peered":"2024-09-15T21:20:03.588199+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:03.588199+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24322048,"num_objects":5938,"num_object_clones":0,"num_object_copies":11876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4494,"num_write_kb":17976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2672}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6030","reported_seq":6140,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:03.601559+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:03.601559+0000","last_peered":"2024-09-15T21:20:03.601559+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:03.601559+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674304,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4429,"num_write_kb":17716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2674}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6211","reported_seq":6334,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.819485+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:06.819485+0000","last_peered":"2024-09-15T21:20:06.819485+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:06.819485+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2776},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6084","reported_seq":6208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.939514+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:06.939514+0000","last_peered":"2024-09-15T21:20:06.939514+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:06.939514+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6084,"log_dups_size":0,"ondisk_log_size":6084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24887296,"num_objects":6076,"num_object_clones":0,"num_object_copies":12152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3669,"num_write_kb":14676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6119","reported_seq":6238,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.939031+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:06.939031+0000","last_peered":"2024-09-15T21:20:06.939031+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:06.939031+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6119,"log_dups_size":0,"ondisk_log_size":6119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047040,"num_objects":6115,"num_object_clones":0,"num_object_copies":12230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3766,"num_write_kb":15064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2750},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6241","reported_seq":6365,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:06.955522+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:06.955522+0000","last_peered":"2024-09-15T21:20:06.955522+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:06.955522+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6241,"log_dups_size":0,"ondisk_log_size":6241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25514601,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3901,"num_write_kb":15604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2760},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399958633,"num_objects":97647,"num_object_clones":0,"num_object_copies":195294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":711,"num_objects_unfound":0,"num_objects_dirty":97647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69142,"num_write_kb":276568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2682,"num_bytes_recovered":10981376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":813051904,"data_stored":812968408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97728,"ondisk_log_size":97728,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953506,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454168,"kb_used_data":301704,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93917672,"statfs":{"total":96636764160,"available":96171696128,"internally_reserved":0,"allocated":308944896,"data_stored":308689446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":156102317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149734,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524912,"kb_used_data":348896,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93846928,"statfs":{"total":96636764160,"available":96099254272,"internally_reserved":0,"allocated":357269504,"data_stored":357014054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287464,"kb_used_data":143320,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94084376,"statfs":{"total":96636764160,"available":96342401024,"internally_reserved":0,"allocated":146759680,"data_stored":146431171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":147582636},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357335040,"data_stored":357331561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146747392,"data_stored":146670854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308969472,"data_stored":308965993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:10.502 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 137 20 99980 99960 2.84944 1.84766 0.00423275 0.0273746 2024-09-15T21:20:11.502 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 138 20 100433 100413 2.84162 1.76953 0.0367491 0.027485 2024-09-15T21:20:12.505 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 139 20 100914 100894 2.83464 1.87891 0.0200605 0.0275546 2024-09-15T21:20:13.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:13.507 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:20:13.506280+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0276137 2024-09-15T21:20:13.507 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:20:13.507 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 140 20 101418 101398 2.82841 1.96875 0.0534114 0.0276137 2024-09-15T21:20:13.942 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:13.942 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:13.983 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-15T21:20:12.350385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409895548,"num_objects":100075,"num_object_clones":0,"num_object_copies":200150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":425,"num_objects_unfound":0,"num_objects_dirty":100075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71570,"num_write_kb":286274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2969,"num_bytes_recovered":12152851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100157,"ondisk_log_size":100157,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1307336,"kb_used_data":815256,"kb_used_omap":12,"kb_used_meta":492019,"kb_avail":281808184,"statfs":{"total":289910292480,"available":288571580416,"internally_reserved":0,"allocated":834822144,"data_stored":833982735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":503827466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,6,13,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27316224,"num_objects":6669,"num_object_clones":0,"num_object_copies":13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-527,"num_objects_unfound":0,"num_objects_dirty":6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6669,"num_write_kb":26676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":2158592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002692"},"pg_stats":[{"pgid":"2.f","version":"44'6245","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.826480+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:11.826480+0000","last_peered":"2024-09-15T21:20:11.826480+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:11.826480+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25579520,"num_objects":6245,"num_object_clones":0,"num_object_copies":12490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6219","reported_seq":6311,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:08.893303+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:08.893303+0000","last_peered":"2024-09-15T21:20:08.893303+0000","last_clean":"2024-09-15T21:20:08.893303+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:08.893303+0000","last_undegraded":"2024-09-15T21:20:08.893303+0000","last_fullsized":"2024-09-15T21:20:08.893303+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6219,"log_dups_size":0,"ondisk_log_size":6219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25448448,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4945,"num_write_kb":19780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6211","reported_seq":12262,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:08.889388+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:08.889388+0000","last_peered":"2024-09-15T21:20:08.889388+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:08.889388+0000","last_undegraded":"2024-09-15T21:20:08.889388+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25423872,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":425,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4934,"num_write_kb":19736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2968,"num_bytes_recovered":12152832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":425},{"shards":"2","objects":5782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6276","reported_seq":6341,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:08.897611+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:08.897611+0000","last_peered":"2024-09-15T21:20:08.897611+0000","last_clean":"2024-09-15T21:20:08.897611+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:08.897611+0000","last_undegraded":"2024-09-15T21:20:08.897611+0000","last_fullsized":"2024-09-15T21:20:08.897611+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25706496,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6276,"num_write_kb":25104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6340","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.798725+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:11.798725+0000","last_peered":"2024-09-15T21:20:11.798725+0000","last_clean":"2024-09-15T21:20:11.798725+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:11.798725+0000","last_undegraded":"2024-09-15T21:20:11.798725+0000","last_fullsized":"2024-09-15T21:20:11.798725+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6340,"log_dups_size":0,"ondisk_log_size":6340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6293","reported_seq":6418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.801291+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:11.801291+0000","last_peered":"2024-09-15T21:20:11.801291+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:11.801291+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3718,"num_write_kb":14872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6467","reported_seq":6585,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.792022+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:11.792022+0000","last_peered":"2024-09-15T21:20:11.792022+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:11.792022+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6261","reported_seq":6376,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.788587+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:11.788587+0000","last_peered":"2024-09-15T21:20:11.788587+0000","last_clean":"2024-09-15T21:20:11.788587+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:11.788587+0000","last_undegraded":"2024-09-15T21:20:11.788587+0000","last_fullsized":"2024-09-15T21:20:11.788587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6116","reported_seq":6236,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:08.883586+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:08.883586+0000","last_peered":"2024-09-15T21:20:08.883586+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:08.883586+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6116,"log_dups_size":0,"ondisk_log_size":6116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":18808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2771}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6202","reported_seq":6320,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.768562+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:11.768562+0000","last_peered":"2024-09-15T21:20:11.768562+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:11.768562+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6202,"log_dups_size":0,"ondisk_log_size":6202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6105","reported_seq":6208,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:08.866060+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:08.866060+0000","last_peered":"2024-09-15T21:20:08.866060+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:08.866060+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6105,"log_dups_size":0,"ondisk_log_size":6105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24985600,"num_objects":6100,"num_object_clones":0,"num_object_copies":12200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2834}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6166","reported_seq":6276,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:08.875034+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:08.875034+0000","last_peered":"2024-09-15T21:20:08.875034+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:08.875034+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6166,"log_dups_size":0,"ondisk_log_size":6166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4565,"num_write_kb":18260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6363","reported_seq":6486,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.775362+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:11.775362+0000","last_peered":"2024-09-15T21:20:11.775362+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:11.775362+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3670,"num_write_kb":14680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2928},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6227","reported_seq":6352,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.780479+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:11.780479+0000","last_peered":"2024-09-15T21:20:11.780479+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:11.780479+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2840},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6277","reported_seq":6396,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.820099+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:11.820099+0000","last_peered":"2024-09-15T21:20:11.820099+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:11.820099+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":15696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6386","reported_seq":6510,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.804960+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:11.804960+0000","last_peered":"2024-09-15T21:20:11.804960+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:11.804960+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26108521,"num_objects":6375,"num_object_clones":0,"num_object_copies":12750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409895529,"num_objects":100073,"num_object_clones":0,"num_object_copies":200146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":425,"num_objects_unfound":0,"num_objects_dirty":100073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71568,"num_write_kb":286272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2968,"num_bytes_recovered":12152832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":834351104,"data_stored":834267608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100154,"ondisk_log_size":100154,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953507,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464676,"kb_used_data":306644,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93907164,"statfs":{"total":96636764160,"available":96160935936,"internally_reserved":0,"allocated":314003456,"data_stored":313748006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536612,"kb_used_data":356436,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93835228,"statfs":{"total":96636764160,"available":96087273472,"internally_reserved":0,"allocated":364990464,"data_stored":364735014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":184479409},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306048,"kb_used_data":152176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94065792,"statfs":{"total":96636764160,"available":96323371008,"internally_reserved":0,"allocated":155828224,"data_stored":155499715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364617728,"data_stored":364614249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155611136,"data_stored":155534598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314122240,"data_stored":314118761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:13.984 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:14.507 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 141 20 101940 101920 2.82281 2.03906 0.0143711 0.0276677 2024-09-15T21:20:14.787 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:14.788 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:14.846 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-15T21:20:14.350885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413667964,"num_objects":100996,"num_object_clones":0,"num_object_copies":201992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":100996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72491,"num_write_kb":289958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3333,"num_bytes_recovered":13643795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101078,"ondisk_log_size":101078,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1316224,"kb_used_data":819984,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281799296,"statfs":{"total":289910292480,"available":288562479104,"internally_reserved":0,"allocated":839663616,"data_stored":838824207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":508087306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,6,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-891,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5874,"num_write_kb":23496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":3649536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002718"},"pg_stats":[{"pgid":"2.f","version":"44'6245","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.826480+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:11.826480+0000","last_peered":"2024-09-15T21:20:11.826480+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:11.826480+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25579520,"num_objects":6245,"num_object_clones":0,"num_object_copies":12490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6373","reported_seq":6465,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.891434+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:13.891434+0000","last_peered":"2024-09-15T21:20:13.891434+0000","last_clean":"2024-09-15T21:20:13.891434+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:13.891434+0000","last_undegraded":"2024-09-15T21:20:13.891434+0000","last_fullsized":"2024-09-15T21:20:13.891434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6373,"log_dups_size":0,"ondisk_log_size":6373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5099,"num_write_kb":20396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6359","reported_seq":13140,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:13.928444+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:13.928444+0000","last_peered":"2024-09-15T21:20:13.928444+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:13.928444+0000","last_undegraded":"2024-09-15T21:20:13.928444+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6359,"log_dups_size":0,"ondisk_log_size":6359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26030080,"num_objects":6355,"num_object_clones":0,"num_object_copies":12710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":6355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3332,"num_bytes_recovered":13643776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":61},{"shards":"2","objects":6294}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6418","reported_seq":6483,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.915494+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:13.915494+0000","last_peered":"2024-09-15T21:20:13.915494+0000","last_clean":"2024-09-15T21:20:13.915494+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:13.915494+0000","last_undegraded":"2024-09-15T21:20:13.915494+0000","last_fullsized":"2024-09-15T21:20:13.915494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6418,"num_write_kb":25672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6340","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.798725+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:11.798725+0000","last_peered":"2024-09-15T21:20:11.798725+0000","last_clean":"2024-09-15T21:20:11.798725+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:11.798725+0000","last_undegraded":"2024-09-15T21:20:11.798725+0000","last_fullsized":"2024-09-15T21:20:11.798725+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6340,"log_dups_size":0,"ondisk_log_size":6340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6293","reported_seq":6418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.801291+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:11.801291+0000","last_peered":"2024-09-15T21:20:11.801291+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:11.801291+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3718,"num_write_kb":14872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6467","reported_seq":6585,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.792022+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:11.792022+0000","last_peered":"2024-09-15T21:20:11.792022+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:11.792022+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6261","reported_seq":6376,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.788587+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:11.788587+0000","last_peered":"2024-09-15T21:20:11.788587+0000","last_clean":"2024-09-15T21:20:11.788587+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:11.788587+0000","last_undegraded":"2024-09-15T21:20:11.788587+0000","last_fullsized":"2024-09-15T21:20:11.788587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6274","reported_seq":6394,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.911476+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:13.911476+0000","last_peered":"2024-09-15T21:20:13.911476+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:13.911476+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4860,"num_write_kb":19440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6202","reported_seq":6320,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.768562+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:11.768562+0000","last_peered":"2024-09-15T21:20:11.768562+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:11.768562+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6202,"log_dups_size":0,"ondisk_log_size":6202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6254","reported_seq":6359,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.901456+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:13.901456+0000","last_peered":"2024-09-15T21:20:13.901456+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:13.901456+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6254,"log_dups_size":0,"ondisk_log_size":6254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2983}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6336","reported_seq":6446,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.926095+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:13.926095+0000","last_peered":"2024-09-15T21:20:13.926095+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:13.926095+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6363","reported_seq":6486,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.775362+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:11.775362+0000","last_peered":"2024-09-15T21:20:11.775362+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:11.775362+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3670,"num_write_kb":14680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2928},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6227","reported_seq":6352,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.780479+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:11.780479+0000","last_peered":"2024-09-15T21:20:11.780479+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:11.780479+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2840},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6277","reported_seq":6396,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.820099+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:11.820099+0000","last_peered":"2024-09-15T21:20:11.820099+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:11.820099+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":15696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6386","reported_seq":6510,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.804960+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:11.804960+0000","last_peered":"2024-09-15T21:20:11.804960+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:11.804960+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26108521,"num_objects":6375,"num_object_clones":0,"num_object_copies":12750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413667945,"num_objects":100994,"num_object_clones":0,"num_object_copies":201988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":100994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72489,"num_write_kb":289956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3332,"num_bytes_recovered":13643776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839368704,"data_stored":839285208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101075,"ondisk_log_size":101075,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473564,"kb_used_data":311372,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93898276,"statfs":{"total":96636764160,"available":96151834624,"internally_reserved":0,"allocated":318844928,"data_stored":318589478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536612,"kb_used_data":356436,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93835228,"statfs":{"total":96636764160,"available":96087273472,"internally_reserved":0,"allocated":364990464,"data_stored":364735014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":184479409},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306048,"kb_used_data":152176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94065792,"statfs":{"total":96636764160,"available":96323371008,"internally_reserved":0,"allocated":155828224,"data_stored":155499715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364617728,"data_stored":364614249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155611136,"data_stored":155534598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319139840,"data_stored":319136361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:14.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:14.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'6245", 'reported_seq': 6317, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.826480+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:11.826480+0000', 'last_peered': '2024-09-15T21:20:11.826480+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:11.826480+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 6245, 'log_dups_size': 0, 'ondisk_log_size': 6245, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25579520, 'num_objects': 6245, 'num_object_clones': 0, 'num_object_copies': 12490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6245, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6245, 'num_write_kb': 24980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2878}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:20:14.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "44'6359", 'reported_seq': 13140, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:13.928444+0000', 'last_change': '2024-09-15T21:18:43.791668+0000', 'last_active': '2024-09-15T21:20:13.928444+0000', 'last_peered': '2024-09-15T21:20:13.928444+0000', 'last_clean': '2024-09-15T21:18:41.353008+0000', 'last_became_active': '2024-09-15T21:18:43.741074+0000', 'last_became_peered': '2024-09-15T21:18:43.741074+0000', 'last_unstale': '2024-09-15T21:20:13.928444+0000', 'last_undegraded': '2024-09-15T21:20:13.928444+0000', 'last_fullsized': '2024-09-15T21:18:43.372668+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1305", 'last_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:06.394090+0000', 'objects_scrubbed': 1273, 'log_size': 6359, 'log_dups_size': 0, 'ondisk_log_size': 6359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 683, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26030080, 'num_objects': 6355, 'num_object_clones': 0, 'num_object_copies': 12710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 61, 'num_objects_unfound': 0, 'num_objects_dirty': 6355, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5082, 'num_write_kb': 20328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3332, 'num_bytes_recovered': 13643776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 61}, {'shards': '2', 'objects': 6294}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'6293", 'reported_seq': 6418, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.801291+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:11.801291+0000', 'last_peered': '2024-09-15T21:20:11.801291+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:11.801291+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 6293, 'log_dups_size': 0, 'ondisk_log_size': 6293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25755648, 'num_objects': 6288, 'num_object_clones': 0, 'num_object_copies': 12576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3718, 'num_write_kb': 14872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2917}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "44'6467", 'reported_seq': 6585, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.792022+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:11.792022+0000', 'last_peered': '2024-09-15T21:20:11.792022+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:11.792022+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 6467, 'log_dups_size': 0, 'ondisk_log_size': 6467, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26464256, 'num_objects': 6461, 'num_object_clones': 0, 'num_object_copies': 12922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4103, 'num_write_kb': 16412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2990}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'6274", 'reported_seq': 6394, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:13.911476+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:13.911476+0000', 'last_peered': '2024-09-15T21:20:13.911476+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:13.911476+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 6274, 'log_dups_size': 0, 'ondisk_log_size': 6274, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25669632, 'num_objects': 6267, 'num_object_clones': 0, 'num_object_copies': 12534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4860, 'num_write_kb': 19440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 2929}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'6202", 'reported_seq': 6320, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.768562+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:11.768562+0000', 'last_peered': '2024-09-15T21:20:11.768562+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:11.768562+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6202, 'log_dups_size': 0, 'ondisk_log_size': 6202, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25370624, 'num_objects': 6194, 'num_object_clones': 0, 'num_object_copies': 12388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3728, 'num_write_kb': 14912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2853}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'6254", 'reported_seq': 6359, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:13.901456+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:13.901456+0000', 'last_peered': '2024-09-15T21:20:13.901456+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:13.901456+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 6254, 'log_dups_size': 0, 'ondisk_log_size': 6254, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25595904, 'num_objects': 6249, 'num_object_clones': 0, 'num_object_copies': 12498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4805, 'num_write_kb': 19220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 2983}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "44'6336", 'reported_seq': 6446, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:13.926095+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:13.926095+0000', 'last_peered': '2024-09-15T21:20:13.926095+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:13.926095+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6336, 'log_dups_size': 0, 'ondisk_log_size': 6336, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25927680, 'num_objects': 6330, 'num_object_clones': 0, 'num_object_copies': 12660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4735, 'num_write_kb': 18940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 2980}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'6363", 'reported_seq': 6486, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.775362+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:11.775362+0000', 'last_peered': '2024-09-15T21:20:11.775362+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:11.775362+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 6363, 'log_dups_size': 0, 'ondisk_log_size': 6363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26054656, 'num_objects': 6361, 'num_object_clones': 0, 'num_object_copies': 12722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6361, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3670, 'num_write_kb': 14680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2928}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'6227", 'reported_seq': 6352, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.780479+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:11.780479+0000', 'last_peered': '2024-09-15T21:20:11.780479+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:11.780479+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6227, 'log_dups_size': 0, 'ondisk_log_size': 6227, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25473024, 'num_objects': 6219, 'num_object_clones': 0, 'num_object_copies': 12438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3812, 'num_write_kb': 15248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2840}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'6277", 'reported_seq': 6396, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.820099+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:11.820099+0000', 'last_peered': '2024-09-15T21:20:11.820099+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:11.820099+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 6277, 'log_dups_size': 0, 'ondisk_log_size': 6277, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25694208, 'num_objects': 6273, 'num_object_clones': 0, 'num_object_copies': 12546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3924, 'num_write_kb': 15696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2908}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "44'6386", 'reported_seq': 6510, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:11.804960+0000', 'last_change': '2024-09-15T21:18:43.771949+0000', 'last_active': '2024-09-15T21:20:11.804960+0000', 'last_peered': '2024-09-15T21:20:11.804960+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:11.804960+0000', 'last_undegraded': '2024-09-15T21:18:43.482860+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 6386, 'log_dups_size': 0, 'ondisk_log_size': 6386, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26108521, 'num_objects': 6375, 'num_object_clones': 0, 'num_object_copies': 12750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4046, 'num_write_kb': 16184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2905}, {'shards': '0,2', 'objects': 3470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:14.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:15.429 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:15.429 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37172/201992 objects degraded (18.403%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100996,"data_bytes":413667964,"bytes_used":1347813376,"bytes_avail":288562479104,"bytes_total":289910292480,"degraded_objects":37172,"degraded_total":201992,"degraded_ratio":0.18402709018178937,"misplaced_objects":61,"misplaced_total":201992,"misplaced_ratio":0.0003019921581052715,"recovering_objects_per_sec":74,"recovering_bytes_per_sec":304059,"recovering_keys_per_sec":0,"num_objects_recovered":891,"num_bytes_recovered":3649536,"num_keys_recovered":0,"write_bytes_sec":2004537,"read_op_per_sec":0,"write_op_per_sec":489},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:15.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:15.459 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:15.507 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 142 20 102329 102309 2.81363 1.51953 0.00753758 0.0277264 2024-09-15T21:20:15.933 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:15.962 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-15T21:20:15.422404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413667964,"num_objects":100996,"num_object_clones":0,"num_object_copies":201992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":100996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72491,"num_write_kb":289958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3333,"num_bytes_recovered":13643795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101078,"ondisk_log_size":101078,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1316224,"kb_used_data":819984,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281799296,"statfs":{"total":289910292480,"available":288562479104,"internally_reserved":0,"allocated":839663616,"data_stored":838824207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":508087306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,6,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-650,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":650,"num_bytes_recovered":2662400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.073715"},"pg_stats":[{"pgid":"2.f","version":"44'6245","reported_seq":6317,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.826480+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:11.826480+0000","last_peered":"2024-09-15T21:20:11.826480+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:11.826480+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25579520,"num_objects":6245,"num_object_clones":0,"num_object_copies":12490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6245,"num_write_kb":24980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2878},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6373","reported_seq":6465,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.891434+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:13.891434+0000","last_peered":"2024-09-15T21:20:13.891434+0000","last_clean":"2024-09-15T21:20:13.891434+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:13.891434+0000","last_undegraded":"2024-09-15T21:20:13.891434+0000","last_fullsized":"2024-09-15T21:20:13.891434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6373,"log_dups_size":0,"ondisk_log_size":6373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5099,"num_write_kb":20396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'6359","reported_seq":13140,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:13.928444+0000","last_change":"2024-09-15T21:18:43.791668+0000","last_active":"2024-09-15T21:20:13.928444+0000","last_peered":"2024-09-15T21:20:13.928444+0000","last_clean":"2024-09-15T21:18:41.353008+0000","last_became_active":"2024-09-15T21:18:43.741074+0000","last_became_peered":"2024-09-15T21:18:43.741074+0000","last_unstale":"2024-09-15T21:20:13.928444+0000","last_undegraded":"2024-09-15T21:20:13.928444+0000","last_fullsized":"2024-09-15T21:18:43.372668+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6359,"log_dups_size":0,"ondisk_log_size":6359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26030080,"num_objects":6355,"num_object_clones":0,"num_object_copies":12710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":6355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3332,"num_bytes_recovered":13643776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":61},{"shards":"2","objects":6294}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'6418","reported_seq":6483,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.915494+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:13.915494+0000","last_peered":"2024-09-15T21:20:13.915494+0000","last_clean":"2024-09-15T21:20:13.915494+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:13.915494+0000","last_undegraded":"2024-09-15T21:20:13.915494+0000","last_fullsized":"2024-09-15T21:20:13.915494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6418,"num_write_kb":25672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'6340","reported_seq":6454,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.798725+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:11.798725+0000","last_peered":"2024-09-15T21:20:11.798725+0000","last_clean":"2024-09-15T21:20:11.798725+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:11.798725+0000","last_undegraded":"2024-09-15T21:20:11.798725+0000","last_fullsized":"2024-09-15T21:20:11.798725+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6340,"log_dups_size":0,"ondisk_log_size":6340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'6293","reported_seq":6418,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.801291+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:11.801291+0000","last_peered":"2024-09-15T21:20:11.801291+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:11.801291+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6293,"log_dups_size":0,"ondisk_log_size":6293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":6288,"num_object_clones":0,"num_object_copies":12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3718,"num_write_kb":14872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2917},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'6467","reported_seq":6585,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.792022+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:11.792022+0000","last_peered":"2024-09-15T21:20:11.792022+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:11.792022+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'6261","reported_seq":6376,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:11.788587+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:11.788587+0000","last_peered":"2024-09-15T21:20:11.788587+0000","last_clean":"2024-09-15T21:20:11.788587+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:11.788587+0000","last_undegraded":"2024-09-15T21:20:11.788587+0000","last_fullsized":"2024-09-15T21:20:11.788587+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25636864,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.396866+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:18:43.396866+0000","last_peered":"2024-09-15T21:18:43.396866+0000","last_clean":"2024-09-15T21:18:43.396866+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:18:43.396866+0000","last_undegraded":"2024-09-15T21:18:43.396866+0000","last_fullsized":"2024-09-15T21:18:43.396866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6274","reported_seq":6394,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.911476+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:13.911476+0000","last_peered":"2024-09-15T21:20:13.911476+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:13.911476+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4860,"num_write_kb":19440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'6202","reported_seq":6320,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.768562+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:11.768562+0000","last_peered":"2024-09-15T21:20:11.768562+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:11.768562+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6202,"log_dups_size":0,"ondisk_log_size":6202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3728,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2853},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":96,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.901431+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:18:43.901431+0000","last_peered":"2024-09-15T21:18:43.901431+0000","last_clean":"2024-09-15T21:18:43.901431+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:18:43.901431+0000","last_undegraded":"2024-09-15T21:18:43.901431+0000","last_fullsized":"2024-09-15T21:18:43.901431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6254","reported_seq":6359,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.901456+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:13.901456+0000","last_peered":"2024-09-15T21:20:13.901456+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:13.901456+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6254,"log_dups_size":0,"ondisk_log_size":6254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2983}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":74,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.469282+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:18:43.469282+0000","last_peered":"2024-09-15T21:18:43.469282+0000","last_clean":"2024-09-15T21:18:43.469282+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:18:43.469282+0000","last_undegraded":"2024-09-15T21:18:43.469282+0000","last_fullsized":"2024-09-15T21:18:43.469282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6336","reported_seq":6446,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.926095+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:13.926095+0000","last_peered":"2024-09-15T21:20:13.926095+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:13.926095+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'6363","reported_seq":6486,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.775362+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:11.775362+0000","last_peered":"2024-09-15T21:20:11.775362+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:11.775362+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6363,"log_dups_size":0,"ondisk_log_size":6363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3670,"num_write_kb":14680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2928},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606495+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:18:43.606495+0000","last_peered":"2024-09-15T21:18:43.606495+0000","last_clean":"2024-09-15T21:18:43.606495+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:18:43.606495+0000","last_undegraded":"2024-09-15T21:18:43.606495+0000","last_fullsized":"2024-09-15T21:18:43.606495+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'6227","reported_seq":6352,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.780479+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:11.780479+0000","last_peered":"2024-09-15T21:20:11.780479+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:11.780479+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6227,"log_dups_size":0,"ondisk_log_size":6227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2840},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'6277","reported_seq":6396,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.820099+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:11.820099+0000","last_peered":"2024-09-15T21:20:11.820099+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:11.820099+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":15696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.606946+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:18:43.606946+0000","last_peered":"2024-09-15T21:18:43.606946+0000","last_clean":"2024-09-15T21:18:43.606946+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:18:43.606946+0000","last_undegraded":"2024-09-15T21:18:43.606946+0000","last_fullsized":"2024-09-15T21:18:43.606946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.382533+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:18:43.382533+0000","last_peered":"2024-09-15T21:18:43.382533+0000","last_clean":"2024-09-15T21:18:43.382533+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:18:43.382533+0000","last_undegraded":"2024-09-15T21:18:43.382533+0000","last_fullsized":"2024-09-15T21:18:43.382533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'6386","reported_seq":6510,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:11.804960+0000","last_change":"2024-09-15T21:18:43.771949+0000","last_active":"2024-09-15T21:20:11.804960+0000","last_peered":"2024-09-15T21:20:11.804960+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:11.804960+0000","last_undegraded":"2024-09-15T21:18:43.482860+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26108521,"num_objects":6375,"num_object_clones":0,"num_object_copies":12750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":16184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":3470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413667945,"num_objects":100994,"num_object_clones":0,"num_object_copies":201988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37172,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":100994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72489,"num_write_kb":289956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3332,"num_bytes_recovered":13643776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839368704,"data_stored":839285208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101075,"ondisk_log_size":101075,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473564,"kb_used_data":311372,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93898276,"statfs":{"total":96636764160,"available":96151834624,"internally_reserved":0,"allocated":318844928,"data_stored":318589478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149735,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536612,"kb_used_data":356436,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93835228,"statfs":{"total":96636764160,"available":96087273472,"internally_reserved":0,"allocated":364990464,"data_stored":364735014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":184479409},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626453,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306048,"kb_used_data":152176,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94065792,"statfs":{"total":96636764160,"available":96323371008,"internally_reserved":0,"allocated":155828224,"data_stored":155499715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":157544108},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364617728,"data_stored":364614249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":155611136,"data_stored":155534598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319139840,"data_stored":319136361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:16.508 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 143 20 102615 102595 2.80175 1.11719 0.0331299 0.0277813 2024-09-15T21:20:17.509 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 144 20 103172 103152 2.7974 2.17578 0.0599279 0.0279191 2024-09-15T21:20:18.509 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 145 20 103749 103729 2.79365 2.25391 0.0176375 0.0279601 2024-09-15T21:20:18.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:19.509 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 146 20 104285 104265 2.78886 2.09375 0.004438 0.0280015 2024-09-15T21:20:19.721 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:19.721 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:19.781 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-15T21:20:18.352080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419816060,"num_objects":102497,"num_object_clones":0,"num_object_copies":204994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":102497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73992,"num_write_kb":295962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3484,"num_bytes_recovered":14262291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102579,"ondisk_log_size":102579,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1347220,"kb_used_data":834328,"kb_used_omap":12,"kb_used_meta":512819,"kb_avail":281768300,"statfs":{"total":289910292480,"available":288530739200,"internally_reserved":0,"allocated":854351872,"data_stored":853491673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":525126666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,2,0,0,0,6,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16015360,"num_objects":3910,"num_object_clones":0,"num_object_copies":7820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":2955,"num_objects_unfound":0,"num_objects_dirty":3910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3910,"num_write_kb":15640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002066"},"pg_stats":[{"pgid":"2.f","version":"46'6407","reported_seq":6483,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.044774+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:17.044774+0000","last_peered":"2024-09-15T21:20:17.044774+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:17.044774+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6407,"log_dups_size":0,"ondisk_log_size":6407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243072,"num_objects":6407,"num_object_clones":0,"num_object_copies":12814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'6373","reported_seq":6465,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.891434+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:13.891434+0000","last_peered":"2024-09-15T21:20:13.891434+0000","last_clean":"2024-09-15T21:20:13.891434+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:13.891434+0000","last_undegraded":"2024-09-15T21:20:13.891434+0000","last_fullsized":"2024-09-15T21:20:13.891434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6373,"log_dups_size":0,"ondisk_log_size":6373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079232,"num_objects":6367,"num_object_clones":0,"num_object_copies":12734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5099,"num_write_kb":20396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6428","reported_seq":13341,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.849157+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:16.849157+0000","last_peered":"2024-09-15T21:20:16.849157+0000","last_clean":"2024-09-15T21:20:16.849157+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:16.849157+0000","last_undegraded":"2024-09-15T21:20:16.849157+0000","last_fullsized":"2024-09-15T21:20:16.849157+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6428,"log_dups_size":0,"ondisk_log_size":6428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26312704,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'6418","reported_seq":6483,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:20:13.915494+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:13.915494+0000","last_peered":"2024-09-15T21:20:13.915494+0000","last_clean":"2024-09-15T21:20:13.915494+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:13.915494+0000","last_undegraded":"2024-09-15T21:20:13.915494+0000","last_fullsized":"2024-09-15T21:20:13.915494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6418,"log_dups_size":0,"ondisk_log_size":6418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6418,"num_write_kb":25672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6501","reported_seq":6619,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:17.062268+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:17.062268+0000","last_peered":"2024-09-15T21:20:17.062268+0000","last_clean":"2024-09-15T21:20:17.062268+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:17.062268+0000","last_undegraded":"2024-09-15T21:20:17.062268+0000","last_fullsized":"2024-09-15T21:20:17.062268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6501,"log_dups_size":0,"ondisk_log_size":6501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6413","reported_seq":6540,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.033027+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:17.033027+0000","last_peered":"2024-09-15T21:20:17.033027+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:17.033027+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3838,"num_write_kb":15352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3037},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6618","reported_seq":6740,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.014396+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:17.014396+0000","last_peered":"2024-09-15T21:20:17.014396+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:17.014396+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":17016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3141},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6382","reported_seq":6501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.922280+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:16.922280+0000","last_peered":"2024-09-15T21:20:16.922280+0000","last_clean":"2024-09-15T21:20:16.922280+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:16.922280+0000","last_undegraded":"2024-09-15T21:20:16.922280+0000","last_fullsized":"2024-09-15T21:20:16.922280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6382,"log_dups_size":0,"ondisk_log_size":6382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'6274","reported_seq":6394,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.911476+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:13.911476+0000","last_peered":"2024-09-15T21:20:13.911476+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:13.911476+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25669632,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4860,"num_write_kb":19440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":2929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.376539+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:18:43.376539+0000","last_peered":"2024-09-15T21:18:43.376539+0000","last_clean":"2024-09-15T21:18:43.376539+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:18:43.376539+0000","last_undegraded":"2024-09-15T21:18:43.376539+0000","last_fullsized":"2024-09-15T21:18:43.376539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6341","reported_seq":6462,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.061568+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:17.061568+0000","last_peered":"2024-09-15T21:20:17.061568+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:17.061568+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6341,"log_dups_size":0,"ondisk_log_size":6341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3867,"num_write_kb":15468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2992},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'6254","reported_seq":6359,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.901456+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:13.901456+0000","last_peered":"2024-09-15T21:20:13.901456+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:13.901456+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6254,"log_dups_size":0,"ondisk_log_size":6254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":2983}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'6336","reported_seq":6446,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:13.926095+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:13.926095+0000","last_peered":"2024-09-15T21:20:13.926095+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:13.926095+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6336,"log_dups_size":0,"ondisk_log_size":6336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":2980}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6506","reported_seq":6633,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.063443+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:17.063443+0000","last_peered":"2024-09-15T21:20:17.063443+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:17.063443+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6506,"log_dups_size":0,"ondisk_log_size":6506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3813,"num_write_kb":15252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3071},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6369","reported_seq":6498,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.048813+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:17.048813+0000","last_peered":"2024-09-15T21:20:17.048813+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:17.048813+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6369,"log_dups_size":0,"ondisk_log_size":6369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3954,"num_write_kb":15816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2982},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:18:43.439496+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:18:43.439496+0000","last_peered":"2024-09-15T21:18:43.439496+0000","last_clean":"2024-09-15T21:18:43.439496+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:18:43.439496+0000","last_undegraded":"2024-09-15T21:18:43.439496+0000","last_fullsized":"2024-09-15T21:18:43.439496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6423","reported_seq":6545,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.070761+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:17.070761+0000","last_peered":"2024-09-15T21:20:17.070761+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:17.070761+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6423,"log_dups_size":0,"ondisk_log_size":6423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3054},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6533","reported_seq":6842,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:17.072852+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:17.072852+0000","last_peered":"2024-09-15T21:20:17.072852+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:17.072852+0000","last_undegraded":"2024-09-15T21:20:17.072852+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710633,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4193,"num_write_kb":16772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":368640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3142},{"shards":"0,2","objects":3380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419816041,"num_objects":102495,"num_object_clones":0,"num_object_copies":204990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":102495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73990,"num_write_kb":295960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3483,"num_bytes_recovered":14262272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":854499328,"data_stored":854415832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102576,"ondisk_log_size":102576,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953508,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473564,"kb_used_data":311372,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93898276,"statfs":{"total":96636764160,"available":96151834624,"internally_reserved":0,"allocated":318844928,"data_stored":318589478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149736,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551448,"kb_used_data":362948,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93820392,"statfs":{"total":96636764160,"available":96072081408,"internally_reserved":0,"allocated":371658752,"data_stored":371392907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":192999089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322208,"kb_used_data":160008,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94049632,"statfs":{"total":96636764160,"available":96306823168,"internally_reserved":0,"allocated":163848192,"data_stored":163509288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":166063788},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,0,0,0,6,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371802112,"data_stored":371798633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163557376,"data_stored":163480838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319139840,"data_stored":319136361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:19.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:20.380 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:20.381 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:20.441 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-15T21:20:20.353810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422933116,"num_objects":103258,"num_object_clones":0,"num_object_copies":206516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":103258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74753,"num_write_kb":299006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3484,"num_bytes_recovered":14262291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103340,"ondisk_log_size":103340,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1355916,"kb_used_data":838864,"kb_used_omap":12,"kb_used_meta":516979,"kb_avail":281759604,"statfs":{"total":289910292480,"available":288521834496,"internally_reserved":0,"allocated":858996736,"data_stored":858126142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":529386506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,0,3,3,1,0,0,6,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13037568,"num_objects":3183,"num_object_clones":0,"num_object_copies":6366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":2955,"num_objects_unfound":0,"num_objects_dirty":3183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003424"},"pg_stats":[{"pgid":"2.f","version":"46'6407","reported_seq":6483,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.044774+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:17.044774+0000","last_peered":"2024-09-15T21:20:17.044774+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:17.044774+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6407,"log_dups_size":0,"ondisk_log_size":6407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243072,"num_objects":6407,"num_object_clones":0,"num_object_copies":12814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6540","reported_seq":6635,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:18.924396+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:18.924396+0000","last_peered":"2024-09-15T21:20:18.924396+0000","last_clean":"2024-09-15T21:20:18.924396+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:18.924396+0000","last_undegraded":"2024-09-15T21:20:18.924396+0000","last_fullsized":"2024-09-15T21:20:18.924396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6540,"log_dups_size":0,"ondisk_log_size":6540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26763264,"num_objects":6534,"num_object_clones":0,"num_object_copies":13068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5266,"num_write_kb":21064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6428","reported_seq":13341,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.849157+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:16.849157+0000","last_peered":"2024-09-15T21:20:16.849157+0000","last_clean":"2024-09-15T21:20:16.849157+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:16.849157+0000","last_undegraded":"2024-09-15T21:20:16.849157+0000","last_fullsized":"2024-09-15T21:20:16.849157+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6428,"log_dups_size":0,"ondisk_log_size":6428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26312704,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6591","reported_seq":6658,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:18.881320+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:18.881320+0000","last_peered":"2024-09-15T21:20:18.881320+0000","last_clean":"2024-09-15T21:20:18.881320+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:18.881320+0000","last_undegraded":"2024-09-15T21:20:18.881320+0000","last_fullsized":"2024-09-15T21:20:18.881320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6591,"log_dups_size":0,"ondisk_log_size":6591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26996736,"num_objects":6591,"num_object_clones":0,"num_object_copies":13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6501","reported_seq":6619,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:17.062268+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:17.062268+0000","last_peered":"2024-09-15T21:20:17.062268+0000","last_clean":"2024-09-15T21:20:17.062268+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:17.062268+0000","last_undegraded":"2024-09-15T21:20:17.062268+0000","last_fullsized":"2024-09-15T21:20:17.062268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6501,"log_dups_size":0,"ondisk_log_size":6501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6413","reported_seq":6540,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.033027+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:17.033027+0000","last_peered":"2024-09-15T21:20:17.033027+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:17.033027+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3838,"num_write_kb":15352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3037},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6618","reported_seq":6740,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.014396+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:17.014396+0000","last_peered":"2024-09-15T21:20:17.014396+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:17.014396+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":17016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3141},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6382","reported_seq":6501,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.922280+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:16.922280+0000","last_peered":"2024-09-15T21:20:16.922280+0000","last_clean":"2024-09-15T21:20:16.922280+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:16.922280+0000","last_undegraded":"2024-09-15T21:20:16.922280+0000","last_fullsized":"2024-09-15T21:20:16.922280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6382,"log_dups_size":0,"ondisk_log_size":6382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6413","reported_seq":6536,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.943330+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:18.943330+0000","last_peered":"2024-09-15T21:20:18.943330+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:18.943330+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6341","reported_seq":6462,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.061568+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:17.061568+0000","last_peered":"2024-09-15T21:20:17.061568+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:17.061568+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6341,"log_dups_size":0,"ondisk_log_size":6341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3867,"num_write_kb":15468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2992},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6387","reported_seq":6496,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.924079+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:18.924079+0000","last_peered":"2024-09-15T21:20:18.924079+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:18.924079+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26140672,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4938,"num_write_kb":19752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6485","reported_seq":6599,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.918747+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:18.918747+0000","last_peered":"2024-09-15T21:20:18.918747+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:18.918747+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6485,"log_dups_size":0,"ondisk_log_size":6485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26537984,"num_objects":6479,"num_object_clones":0,"num_object_copies":12958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4884,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3129}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6506","reported_seq":6633,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.063443+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:17.063443+0000","last_peered":"2024-09-15T21:20:17.063443+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:17.063443+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6506,"log_dups_size":0,"ondisk_log_size":6506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3813,"num_write_kb":15252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3071},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6369","reported_seq":6498,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.048813+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:17.048813+0000","last_peered":"2024-09-15T21:20:17.048813+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:17.048813+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6369,"log_dups_size":0,"ondisk_log_size":6369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3954,"num_write_kb":15816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2982},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6423","reported_seq":6545,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:17.070761+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:17.070761+0000","last_peered":"2024-09-15T21:20:17.070761+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:17.070761+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6423,"log_dups_size":0,"ondisk_log_size":6423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3054},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6533","reported_seq":6842,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:17.072852+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:17.072852+0000","last_peered":"2024-09-15T21:20:17.072852+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:17.072852+0000","last_undegraded":"2024-09-15T21:20:17.072852+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710633,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4193,"num_write_kb":16772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":368640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3142},{"shards":"0,2","objects":3380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422933097,"num_objects":103256,"num_object_clones":0,"num_object_copies":206512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":103256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74751,"num_write_kb":299004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3483,"num_bytes_recovered":14262272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":859328512,"data_stored":859245016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103337,"ondisk_log_size":103337,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953509,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482260,"kb_used_data":315908,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93889580,"statfs":{"total":96636764160,"available":96142929920,"internally_reserved":0,"allocated":323489792,"data_stored":323223947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":170323629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149736,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551448,"kb_used_data":362948,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93820392,"statfs":{"total":96636764160,"available":96072081408,"internally_reserved":0,"allocated":371658752,"data_stored":371392907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":192999089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322208,"kb_used_data":160008,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94049632,"statfs":{"total":96636764160,"available":96306823168,"internally_reserved":0,"allocated":163848192,"data_stored":163509288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":166063788},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,0,0,0,6,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371802112,"data_stored":371798633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163557376,"data_stored":163480838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323969024,"data_stored":323965545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'6407", 'reported_seq': 6483, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.044774+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:17.044774+0000', 'last_peered': '2024-09-15T21:20:17.044774+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:17.044774+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 6407, 'log_dups_size': 0, 'ondisk_log_size': 6407, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26243072, 'num_objects': 6407, 'num_object_clones': 0, 'num_object_copies': 12814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6407, 'num_write_kb': 25628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3040}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'6413", 'reported_seq': 6540, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.033027+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:17.033027+0000', 'last_peered': '2024-09-15T21:20:17.033027+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:17.033027+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 6413, 'log_dups_size': 0, 'ondisk_log_size': 6413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26247168, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3838, 'num_write_kb': 15352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3037}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'6618", 'reported_seq': 6740, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.014396+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:17.014396+0000', 'last_peered': '2024-09-15T21:20:17.014396+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:17.014396+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 6618, 'log_dups_size': 0, 'ondisk_log_size': 6618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27082752, 'num_objects': 6612, 'num_object_clones': 0, 'num_object_copies': 13224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4254, 'num_write_kb': 17016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3141}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'6413", 'reported_seq': 6536, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:18.943330+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:18.943330+0000', 'last_peered': '2024-09-15T21:20:18.943330+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:18.943330+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 6413, 'log_dups_size': 0, 'ondisk_log_size': 6413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26238976, 'num_objects': 6406, 'num_object_clones': 0, 'num_object_copies': 12812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4999, 'num_write_kb': 19996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 3068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'6341", 'reported_seq': 6462, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.061568+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:17.061568+0000', 'last_peered': '2024-09-15T21:20:17.061568+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:17.061568+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6341, 'log_dups_size': 0, 'ondisk_log_size': 6341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25939968, 'num_objects': 6333, 'num_object_clones': 0, 'num_object_copies': 12666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3867, 'num_write_kb': 15468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2992}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'6387", 'reported_seq': 6496, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:18.924079+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:18.924079+0000', 'last_peered': '2024-09-15T21:20:18.924079+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:18.924079+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 6387, 'log_dups_size': 0, 'ondisk_log_size': 6387, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26140672, 'num_objects': 6382, 'num_object_clones': 0, 'num_object_copies': 12764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6382, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4938, 'num_write_kb': 19752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 3116}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'6485", 'reported_seq': 6599, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:18.918747+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:18.918747+0000', 'last_peered': '2024-09-15T21:20:18.918747+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:18.918747+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6485, 'log_dups_size': 0, 'ondisk_log_size': 6485, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26537984, 'num_objects': 6479, 'num_object_clones': 0, 'num_object_copies': 12958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4884, 'num_write_kb': 19536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 3129}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'6506", 'reported_seq': 6633, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.063443+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:17.063443+0000', 'last_peered': '2024-09-15T21:20:17.063443+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:17.063443+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 6506, 'log_dups_size': 0, 'ondisk_log_size': 6506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26640384, 'num_objects': 6504, 'num_object_clones': 0, 'num_object_copies': 13008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3813, 'num_write_kb': 15252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3071}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:20.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'6369", 'reported_seq': 6498, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.048813+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:17.048813+0000', 'last_peered': '2024-09-15T21:20:17.048813+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:17.048813+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6369, 'log_dups_size': 0, 'ondisk_log_size': 6369, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26054656, 'num_objects': 6361, 'num_object_clones': 0, 'num_object_copies': 12722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6361, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3954, 'num_write_kb': 15816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2982}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:20.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'6423", 'reported_seq': 6545, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:17.070761+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:17.070761+0000', 'last_peered': '2024-09-15T21:20:17.070761+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:17.070761+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 6423, 'log_dups_size': 0, 'ondisk_log_size': 6423, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26292224, 'num_objects': 6419, 'num_object_clones': 0, 'num_object_copies': 12838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4070, 'num_write_kb': 16280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3054}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:20.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'6533", 'reported_seq': 6842, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:17.072852+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:17.072852+0000', 'last_peered': '2024-09-15T21:20:17.072852+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:17.072852+0000', 'last_undegraded': '2024-09-15T21:20:17.072852+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 6533, 'log_dups_size': 0, 'ondisk_log_size': 6533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26710633, 'num_objects': 6522, 'num_object_clones': 0, 'num_object_copies': 13044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3380, 'num_objects_unfound': 0, 'num_objects_dirty': 6522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4193, 'num_write_kb': 16772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 90, 'num_bytes_recovered': 368640, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3142}, {'shards': '0,2', 'objects': 3380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:20.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:20.510 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 147 20 104789 104769 2.78326 1.96875 0.0135231 0.0280596 2024-09-15T21:20:21.511 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 148 20 105323 105303 2.77853 2.08594 0.00999936 0.0281083 2024-09-15T21:20:21.612 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:21.612 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/206516 objects degraded (16.318%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103258,"data_bytes":422933116,"bytes_used":1388457984,"bytes_avail":288521834496,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":206516,"degraded_ratio":0.16317863991167755,"misplaced_objects":3380,"misplaced_total":206516,"misplaced_ratio":0.01636677061341494,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":263567,"recovering_keys_per_sec":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"write_bytes_sec":1628998,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:21.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:21.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:22.511 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 149 20 105846 105826 2.77359 2.04297 0.0527577 0.0281598 2024-09-15T21:20:22.539 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:22.540 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:22.608 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-15T21:20:22.354240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":430662268,"num_objects":105145,"num_object_clones":0,"num_object_copies":210290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76640,"num_write_kb":306554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":14647315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105227,"ondisk_log_size":105227,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1382100,"kb_used_data":855332,"kb_used_omap":12,"kb_used_meta":526707,"kb_avail":281733420,"statfs":{"total":289910292480,"available":288495022080,"internally_reserved":0,"allocated":875859968,"data_stored":874989374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":539347978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,4,9,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":3225,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4149,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003355"},"pg_stats":[{"pgid":"2.f","version":"46'6580","reported_seq":6656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.097036+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:22.097036+0000","last_peered":"2024-09-15T21:20:22.097036+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:22.097036+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3213},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6540","reported_seq":6635,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:18.924396+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:18.924396+0000","last_peered":"2024-09-15T21:20:18.924396+0000","last_clean":"2024-09-15T21:20:18.924396+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:18.924396+0000","last_undegraded":"2024-09-15T21:20:18.924396+0000","last_fullsized":"2024-09-15T21:20:18.924396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6540,"log_dups_size":0,"ondisk_log_size":6540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26763264,"num_objects":6534,"num_object_clones":0,"num_object_copies":13068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5266,"num_write_kb":21064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6597","reported_seq":13511,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:21.864737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:21.864737+0000","last_peered":"2024-09-15T21:20:21.864737+0000","last_clean":"2024-09-15T21:20:21.864737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:21.864737+0000","last_undegraded":"2024-09-15T21:20:21.864737+0000","last_fullsized":"2024-09-15T21:20:21.864737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27004928,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6591","reported_seq":6658,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:18.881320+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:18.881320+0000","last_peered":"2024-09-15T21:20:18.881320+0000","last_clean":"2024-09-15T21:20:18.881320+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:18.881320+0000","last_undegraded":"2024-09-15T21:20:18.881320+0000","last_fullsized":"2024-09-15T21:20:18.881320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6591,"log_dups_size":0,"ondisk_log_size":6591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26996736,"num_objects":6591,"num_object_clones":0,"num_object_copies":13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6679","reported_seq":6797,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.028402+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:22.028402+0000","last_peered":"2024-09-15T21:20:22.028402+0000","last_clean":"2024-09-15T21:20:22.028402+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:22.028402+0000","last_undegraded":"2024-09-15T21:20:22.028402+0000","last_fullsized":"2024-09-15T21:20:22.028402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6578","reported_seq":6705,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.100272+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:22.100272+0000","last_peered":"2024-09-15T21:20:22.100272+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:22.100272+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4003,"num_write_kb":16012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6796","reported_seq":6918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.127559+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:22.127559+0000","last_peered":"2024-09-15T21:20:22.127559+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:22.127559+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6796,"log_dups_size":0,"ondisk_log_size":6796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6568","reported_seq":6687,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.092945+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:22.092945+0000","last_peered":"2024-09-15T21:20:22.092945+0000","last_clean":"2024-09-15T21:20:22.092945+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:22.092945+0000","last_undegraded":"2024-09-15T21:20:22.092945+0000","last_fullsized":"2024-09-15T21:20:22.092945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4353,"num_write_kb":17412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6413","reported_seq":6536,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.943330+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:18.943330+0000","last_peered":"2024-09-15T21:20:18.943330+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:18.943330+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6526","reported_seq":6648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.101161+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:22.101161+0000","last_peered":"2024-09-15T21:20:22.101161+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:22.101161+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6526,"log_dups_size":0,"ondisk_log_size":6526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26697728,"num_objects":6518,"num_object_clones":0,"num_object_copies":13036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6387","reported_seq":6496,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.924079+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:18.924079+0000","last_peered":"2024-09-15T21:20:18.924079+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:18.924079+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26140672,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4938,"num_write_kb":19752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6485","reported_seq":6599,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:18.918747+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:18.918747+0000","last_peered":"2024-09-15T21:20:18.918747+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:18.918747+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6485,"log_dups_size":0,"ondisk_log_size":6485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26537984,"num_objects":6479,"num_object_clones":0,"num_object_copies":12958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4884,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3129}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6676","reported_seq":6803,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.148471+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:22.148471+0000","last_peered":"2024-09-15T21:20:22.148471+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:22.148471+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27336704,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3983,"num_write_kb":15932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6523","reported_seq":6652,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.082259+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:22.082259+0000","last_peered":"2024-09-15T21:20:22.082259+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:22.082259+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6523,"log_dups_size":0,"ondisk_log_size":6523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6589","reported_seq":6712,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:21.983253+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:21.983253+0000","last_peered":"2024-09-15T21:20:21.983253+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:21.983253+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4236,"num_write_kb":16944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6696","reported_seq":7193,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:22.135281+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:22.135281+0000","last_peered":"2024-09-15T21:20:22.135281+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:22.135281+0000","last_undegraded":"2024-09-15T21:20:22.135281+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27378281,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3399},{"shards":"0,2","objects":3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":430662249,"num_objects":105143,"num_object_clones":0,"num_object_copies":210286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76638,"num_write_kb":306552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3577,"num_bytes_recovered":14647296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":876503040,"data_stored":876419544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105224,"ondisk_log_size":105224,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953509,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482260,"kb_used_data":315908,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93889580,"statfs":{"total":96636764160,"available":96142929920,"internally_reserved":0,"allocated":323489792,"data_stored":323223947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":170323629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547152,"kb_used_data":371136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93824688,"statfs":{"total":96636764160,"available":96076480512,"internally_reserved":0,"allocated":380043264,"data_stored":379777419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,1,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352688,"kb_used_data":168288,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":94019152,"statfs":{"total":96636764160,"available":96275611648,"internally_reserved":0,"allocated":172326912,"data_stored":171988008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":380313600,"data_stored":380310121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172143878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323969024,"data_stored":323965545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:23.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 150 20 106393 106373 2.76931 2.13672 0.0212978 0.0282016 2024-09-15T21:20:24.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 151 20 106947 106927 2.7653 2.16406 0.039252 0.0282432 2024-09-15T21:20:25.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 152 20 107510 107490 2.76158 2.19922 0.00775762 0.0282823 2024-09-15T21:20:25.610 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:26.293 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:26.293 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:26.321 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-15T21:20:24.354741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433963644,"num_objects":105951,"num_object_clones":0,"num_object_copies":211902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77446,"num_write_kb":309778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":14647315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106033,"ondisk_log_size":106033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1374920,"kb_used_data":860632,"kb_used_omap":12,"kb_used_meta":514227,"kb_avail":281740600,"statfs":{"total":289910292480,"available":288502374400,"internally_reserved":0,"allocated":881287168,"data_stored":880416574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":526568458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,5,6,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20295680,"num_objects":4955,"num_object_clones":0,"num_object_copies":9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":3225,"num_objects_unfound":0,"num_objects_dirty":4955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4955,"num_write_kb":19820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.932337"},"pg_stats":[{"pgid":"2.f","version":"46'6580","reported_seq":6656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.097036+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:22.097036+0000","last_peered":"2024-09-15T21:20:22.097036+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:22.097036+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3213},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6717","reported_seq":6813,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.812400+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:23.812400+0000","last_peered":"2024-09-15T21:20:23.812400+0000","last_clean":"2024-09-15T21:20:23.812400+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:23.812400+0000","last_undegraded":"2024-09-15T21:20:23.812400+0000","last_fullsized":"2024-09-15T21:20:23.812400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6717,"log_dups_size":0,"ondisk_log_size":6717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6597","reported_seq":13511,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:21.864737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:21.864737+0000","last_peered":"2024-09-15T21:20:21.864737+0000","last_clean":"2024-09-15T21:20:21.864737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:21.864737+0000","last_undegraded":"2024-09-15T21:20:21.864737+0000","last_fullsized":"2024-09-15T21:20:21.864737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27004928,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6746","reported_seq":6815,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.938061+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:23.938061+0000","last_peered":"2024-09-15T21:20:23.938061+0000","last_clean":"2024-09-15T21:20:23.938061+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:23.938061+0000","last_undegraded":"2024-09-15T21:20:23.938061+0000","last_fullsized":"2024-09-15T21:20:23.938061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6746,"log_dups_size":0,"ondisk_log_size":6746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27631616,"num_objects":6746,"num_object_clones":0,"num_object_copies":13492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6679","reported_seq":6797,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.028402+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:22.028402+0000","last_peered":"2024-09-15T21:20:22.028402+0000","last_clean":"2024-09-15T21:20:22.028402+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:22.028402+0000","last_undegraded":"2024-09-15T21:20:22.028402+0000","last_fullsized":"2024-09-15T21:20:22.028402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6578","reported_seq":6705,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.100272+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:22.100272+0000","last_peered":"2024-09-15T21:20:22.100272+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:22.100272+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4003,"num_write_kb":16012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6796","reported_seq":6918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.127559+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:22.127559+0000","last_peered":"2024-09-15T21:20:22.127559+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:22.127559+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6796,"log_dups_size":0,"ondisk_log_size":6796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6568","reported_seq":6687,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.092945+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:22.092945+0000","last_peered":"2024-09-15T21:20:22.092945+0000","last_clean":"2024-09-15T21:20:22.092945+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:22.092945+0000","last_undegraded":"2024-09-15T21:20:22.092945+0000","last_fullsized":"2024-09-15T21:20:22.092945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4353,"num_write_kb":17412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6580","reported_seq":6704,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.909332+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:23.909332+0000","last_peered":"2024-09-15T21:20:23.909332+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:23.909332+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6526","reported_seq":6648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.101161+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:22.101161+0000","last_peered":"2024-09-15T21:20:22.101161+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:22.101161+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6526,"log_dups_size":0,"ondisk_log_size":6526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26697728,"num_objects":6518,"num_object_clones":0,"num_object_copies":13036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6550","reported_seq":6659,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.874183+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:23.874183+0000","last_peered":"2024-09-15T21:20:23.874183+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:23.874183+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26808320,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5101,"num_write_kb":20404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6629","reported_seq":6743,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.917803+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:23.917803+0000","last_peered":"2024-09-15T21:20:23.917803+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:23.917803+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6629,"log_dups_size":0,"ondisk_log_size":6629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6676","reported_seq":6803,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.148471+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:22.148471+0000","last_peered":"2024-09-15T21:20:22.148471+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:22.148471+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27336704,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3983,"num_write_kb":15932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6523","reported_seq":6652,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.082259+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:22.082259+0000","last_peered":"2024-09-15T21:20:22.082259+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:22.082259+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6523,"log_dups_size":0,"ondisk_log_size":6523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6589","reported_seq":6712,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:21.983253+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:21.983253+0000","last_peered":"2024-09-15T21:20:21.983253+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:21.983253+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4236,"num_write_kb":16944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6696","reported_seq":7193,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:22.135281+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:22.135281+0000","last_peered":"2024-09-15T21:20:22.135281+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:22.135281+0000","last_undegraded":"2024-09-15T21:20:22.135281+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27378281,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3399},{"shards":"0,2","objects":3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433963625,"num_objects":105949,"num_object_clones":0,"num_object_copies":211898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77444,"num_write_kb":309776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3577,"num_bytes_recovered":14647296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":881856512,"data_stored":881773016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106030,"ondisk_log_size":106030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475080,"kb_used_data":321208,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93896760,"statfs":{"total":96636764160,"available":96150282240,"internally_reserved":0,"allocated":328916992,"data_stored":328651147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157544109},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547152,"kb_used_data":371136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93824688,"statfs":{"total":96636764160,"available":96076480512,"internally_reserved":0,"allocated":380043264,"data_stored":379777419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,1,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352688,"kb_used_data":168288,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":94019152,"statfs":{"total":96636764160,"available":96275611648,"internally_reserved":0,"allocated":172326912,"data_stored":171988008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":380313600,"data_stored":380310121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172143878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329322496,"data_stored":329319017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:26.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:26.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 153 20 107991 107971 2.75581 1.87891 0.0649879 0.02834 2024-09-15T21:20:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:27.095 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:27.112 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-15T21:20:26.355210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433963644,"num_objects":105951,"num_object_clones":0,"num_object_copies":211902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77446,"num_write_kb":309778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3578,"num_bytes_recovered":14647315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106033,"ondisk_log_size":106033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1374920,"kb_used_data":860632,"kb_used_omap":12,"kb_used_meta":514227,"kb_avail":281740600,"statfs":{"total":289910292480,"available":288502374400,"internally_reserved":0,"allocated":881287168,"data_stored":880416574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":526568458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,5,6,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20295680,"num_objects":4955,"num_object_clones":0,"num_object_copies":9910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":3225,"num_objects_unfound":0,"num_objects_dirty":4955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4955,"num_write_kb":19820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003797"},"pg_stats":[{"pgid":"2.f","version":"46'6580","reported_seq":6656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.097036+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:22.097036+0000","last_peered":"2024-09-15T21:20:22.097036+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:22.097036+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3213},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6717","reported_seq":6813,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.812400+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:23.812400+0000","last_peered":"2024-09-15T21:20:23.812400+0000","last_clean":"2024-09-15T21:20:23.812400+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:23.812400+0000","last_undegraded":"2024-09-15T21:20:23.812400+0000","last_fullsized":"2024-09-15T21:20:23.812400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6717,"log_dups_size":0,"ondisk_log_size":6717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6597","reported_seq":13511,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:21.864737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:21.864737+0000","last_peered":"2024-09-15T21:20:21.864737+0000","last_clean":"2024-09-15T21:20:21.864737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:21.864737+0000","last_undegraded":"2024-09-15T21:20:21.864737+0000","last_fullsized":"2024-09-15T21:20:21.864737+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6597,"log_dups_size":0,"ondisk_log_size":6597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27004928,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5320,"num_write_kb":21280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6746","reported_seq":6815,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.938061+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:23.938061+0000","last_peered":"2024-09-15T21:20:23.938061+0000","last_clean":"2024-09-15T21:20:23.938061+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:23.938061+0000","last_undegraded":"2024-09-15T21:20:23.938061+0000","last_fullsized":"2024-09-15T21:20:23.938061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6746,"log_dups_size":0,"ondisk_log_size":6746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27631616,"num_objects":6746,"num_object_clones":0,"num_object_copies":13492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6679","reported_seq":6797,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.028402+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:22.028402+0000","last_peered":"2024-09-15T21:20:22.028402+0000","last_clean":"2024-09-15T21:20:22.028402+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:22.028402+0000","last_undegraded":"2024-09-15T21:20:22.028402+0000","last_fullsized":"2024-09-15T21:20:22.028402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6578","reported_seq":6705,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.100272+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:22.100272+0000","last_peered":"2024-09-15T21:20:22.100272+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:22.100272+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6578,"log_dups_size":0,"ondisk_log_size":6578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4003,"num_write_kb":16012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6796","reported_seq":6918,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.127559+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:22.127559+0000","last_peered":"2024-09-15T21:20:22.127559+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:22.127559+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6796,"log_dups_size":0,"ondisk_log_size":6796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3319},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6568","reported_seq":6687,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:22.092945+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:22.092945+0000","last_peered":"2024-09-15T21:20:22.092945+0000","last_clean":"2024-09-15T21:20:22.092945+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:22.092945+0000","last_undegraded":"2024-09-15T21:20:22.092945+0000","last_fullsized":"2024-09-15T21:20:22.092945+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4353,"num_write_kb":17412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6580","reported_seq":6704,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.909332+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:23.909332+0000","last_peered":"2024-09-15T21:20:23.909332+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:23.909332+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6526","reported_seq":6648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.101161+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:22.101161+0000","last_peered":"2024-09-15T21:20:22.101161+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:22.101161+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6526,"log_dups_size":0,"ondisk_log_size":6526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26697728,"num_objects":6518,"num_object_clones":0,"num_object_copies":13036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6550","reported_seq":6659,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.874183+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:23.874183+0000","last_peered":"2024-09-15T21:20:23.874183+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:23.874183+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26808320,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5101,"num_write_kb":20404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6629","reported_seq":6743,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.917803+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:23.917803+0000","last_peered":"2024-09-15T21:20:23.917803+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:23.917803+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6629,"log_dups_size":0,"ondisk_log_size":6629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6676","reported_seq":6803,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.148471+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:22.148471+0000","last_peered":"2024-09-15T21:20:22.148471+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:22.148471+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27336704,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3983,"num_write_kb":15932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3241},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6523","reported_seq":6652,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:22.082259+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:22.082259+0000","last_peered":"2024-09-15T21:20:22.082259+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:22.082259+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6523,"log_dups_size":0,"ondisk_log_size":6523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4108,"num_write_kb":16432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6589","reported_seq":6712,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:21.983253+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:21.983253+0000","last_peered":"2024-09-15T21:20:21.983253+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:21.983253+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6589,"log_dups_size":0,"ondisk_log_size":6589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4236,"num_write_kb":16944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6696","reported_seq":7193,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:22.135281+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:22.135281+0000","last_peered":"2024-09-15T21:20:22.135281+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:22.135281+0000","last_undegraded":"2024-09-15T21:20:22.135281+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27378281,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4356,"num_write_kb":17424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3399},{"shards":"0,2","objects":3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433963625,"num_objects":105949,"num_object_clones":0,"num_object_copies":211898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3286,"num_objects_unfound":0,"num_objects_dirty":105949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77444,"num_write_kb":309776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3577,"num_bytes_recovered":14647296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":881856512,"data_stored":881773016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106030,"ondisk_log_size":106030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475080,"kb_used_data":321208,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93896760,"statfs":{"total":96636764160,"available":96150282240,"internally_reserved":0,"allocated":328916992,"data_stored":328651147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157544109},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149737,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547152,"kb_used_data":371136,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93824688,"statfs":{"total":96636764160,"available":96076480512,"internally_reserved":0,"allocated":380043264,"data_stored":379777419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":180219569},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,1,1,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352688,"kb_used_data":168288,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":94019152,"statfs":{"total":96636764160,"available":96275611648,"internally_reserved":0,"allocated":172326912,"data_stored":171988008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":380313600,"data_stored":380310121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172220416,"data_stored":172143878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329322496,"data_stored":329319017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'6580", 'reported_seq': 6656, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.097036+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:22.097036+0000', 'last_peered': '2024-09-15T21:20:22.097036+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:22.097036+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 6580, 'log_dups_size': 0, 'ondisk_log_size': 6580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26951680, 'num_objects': 6580, 'num_object_clones': 0, 'num_object_copies': 13160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6580, 'num_write_kb': 26320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3213}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'6578", 'reported_seq': 6705, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.100272+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:22.100272+0000', 'last_peered': '2024-09-15T21:20:22.100272+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:22.100272+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 6578, 'log_dups_size': 0, 'ondisk_log_size': 6578, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26923008, 'num_objects': 6573, 'num_object_clones': 0, 'num_object_copies': 13146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4003, 'num_write_kb': 16012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3202}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:27.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'6796", 'reported_seq': 6918, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.127559+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:22.127559+0000', 'last_peered': '2024-09-15T21:20:22.127559+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:22.127559+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 6796, 'log_dups_size': 0, 'ondisk_log_size': 6796, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27811840, 'num_objects': 6790, 'num_object_clones': 0, 'num_object_copies': 13580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4432, 'num_write_kb': 17728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3319}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'6580", 'reported_seq': 6704, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:23.909332+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:23.909332+0000', 'last_peered': '2024-09-15T21:20:23.909332+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:23.909332+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 6580, 'log_dups_size': 0, 'ondisk_log_size': 6580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26923008, 'num_objects': 6573, 'num_object_clones': 0, 'num_object_copies': 13146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5166, 'num_write_kb': 20664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 3235}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'6526", 'reported_seq': 6648, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.101161+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:22.101161+0000', 'last_peered': '2024-09-15T21:20:22.101161+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:22.101161+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6526, 'log_dups_size': 0, 'ondisk_log_size': 6526, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26697728, 'num_objects': 6518, 'num_object_clones': 0, 'num_object_copies': 13036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4052, 'num_write_kb': 16208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3177}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'6550", 'reported_seq': 6659, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:23.874183+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:23.874183+0000', 'last_peered': '2024-09-15T21:20:23.874183+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:23.874183+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 6550, 'log_dups_size': 0, 'ondisk_log_size': 6550, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26808320, 'num_objects': 6545, 'num_object_clones': 0, 'num_object_copies': 13090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5101, 'num_write_kb': 20404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 3279}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'6629", 'reported_seq': 6743, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:23.917803+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:23.917803+0000', 'last_peered': '2024-09-15T21:20:23.917803+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:23.917803+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6629, 'log_dups_size': 0, 'ondisk_log_size': 6629, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27127808, 'num_objects': 6623, 'num_object_clones': 0, 'num_object_copies': 13246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5028, 'num_write_kb': 20112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 3273}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'6676", 'reported_seq': 6803, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.148471+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:22.148471+0000', 'last_peered': '2024-09-15T21:20:22.148471+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:22.148471+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 6676, 'log_dups_size': 0, 'ondisk_log_size': 6676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27336704, 'num_objects': 6674, 'num_object_clones': 0, 'num_object_copies': 13348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3983, 'num_write_kb': 15932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3241}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:27.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'6523", 'reported_seq': 6652, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:22.082259+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:22.082259+0000', 'last_peered': '2024-09-15T21:20:22.082259+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:22.082259+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6523, 'log_dups_size': 0, 'ondisk_log_size': 6523, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26685440, 'num_objects': 6515, 'num_object_clones': 0, 'num_object_copies': 13030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4108, 'num_write_kb': 16432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3136}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:27.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'6589", 'reported_seq': 6712, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:21.983253+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:21.983253+0000', 'last_peered': '2024-09-15T21:20:21.983253+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:21.983253+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 6589, 'log_dups_size': 0, 'ondisk_log_size': 6589, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26972160, 'num_objects': 6585, 'num_object_clones': 0, 'num_object_copies': 13170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4236, 'num_write_kb': 16944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3220}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:27.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'6696", 'reported_seq': 7193, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:22.135281+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:22.135281+0000', 'last_peered': '2024-09-15T21:20:22.135281+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:22.135281+0000', 'last_undegraded': '2024-09-15T21:20:22.135281+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 6696, 'log_dups_size': 0, 'ondisk_log_size': 6696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27378281, 'num_objects': 6685, 'num_object_clones': 0, 'num_object_copies': 13370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3286, 'num_objects_unfound': 0, 'num_objects_dirty': 6685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4356, 'num_write_kb': 17424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 184, 'num_bytes_recovered': 753664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3399}, {'shards': '0,2', 'objects': 3286}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:27.117 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:27.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 154 20 108407 108387 2.74847 1.625 0.0279566 0.0284169 2024-09-15T21:20:27.982 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:27.982 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/211902 objects degraded (15.903%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":241,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105951,"data_bytes":433963644,"bytes_used":1407918080,"bytes_avail":288502374400,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":211902,"degraded_ratio":0.1590310615284424,"misplaced_objects":3286,"misplaced_total":211902,"misplaced_ratio":0.01550716840803768,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100313,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"write_bytes_sec":2028797,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:28.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:28.017 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:28.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 155 20 108928 108908 2.74387 2.03516 0.082811 0.0284667 2024-09-15T21:20:28.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:28.840 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:28.921 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-15T21:20:28.357590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441373308,"num_objects":107760,"num_object_clones":0,"num_object_copies":215520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3181,"num_objects_unfound":0,"num_objects_dirty":107760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79255,"num_write_kb":317014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":15077395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107842,"ondisk_log_size":107842,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1409104,"kb_used_data":876768,"kb_used_omap":12,"kb_used_meta":532275,"kb_avail":281706416,"statfs":{"total":289910292480,"available":288467369984,"internally_reserved":0,"allocated":897810432,"data_stored":896939838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":545049610},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,11,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27705344,"num_objects":6764,"num_object_clones":0,"num_object_copies":13528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3473,"num_objects_misplaced":3120,"num_objects_unfound":0,"num_objects_dirty":6764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6764,"num_write_kb":27056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.933639"},"pg_stats":[{"pgid":"2.f","version":"46'6741","reported_seq":6817,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:27.026280+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:27.026280+0000","last_peered":"2024-09-15T21:20:27.026280+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:27.026280+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6741,"log_dups_size":0,"ondisk_log_size":6741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6717","reported_seq":6813,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.812400+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:23.812400+0000","last_peered":"2024-09-15T21:20:23.812400+0000","last_clean":"2024-09-15T21:20:23.812400+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:23.812400+0000","last_undegraded":"2024-09-15T21:20:23.812400+0000","last_fullsized":"2024-09-15T21:20:23.812400+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6717,"log_dups_size":0,"ondisk_log_size":6717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6771","reported_seq":13685,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:26.847221+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:26.847221+0000","last_peered":"2024-09-15T21:20:26.847221+0000","last_clean":"2024-09-15T21:20:26.847221+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:26.847221+0000","last_undegraded":"2024-09-15T21:20:26.847221+0000","last_fullsized":"2024-09-15T21:20:26.847221+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6771,"log_dups_size":0,"ondisk_log_size":6771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5494,"num_write_kb":21976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6746","reported_seq":6815,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:23.938061+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:23.938061+0000","last_peered":"2024-09-15T21:20:23.938061+0000","last_clean":"2024-09-15T21:20:23.938061+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:23.938061+0000","last_undegraded":"2024-09-15T21:20:23.938061+0000","last_fullsized":"2024-09-15T21:20:23.938061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6746,"log_dups_size":0,"ondisk_log_size":6746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27631616,"num_objects":6746,"num_object_clones":0,"num_object_copies":13492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'6859","reported_seq":6977,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:26.913303+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:26.913303+0000","last_peered":"2024-09-15T21:20:26.913303+0000","last_clean":"2024-09-15T21:20:26.913303+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:26.913303+0000","last_undegraded":"2024-09-15T21:20:26.913303+0000","last_fullsized":"2024-09-15T21:20:26.913303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":6859,"log_dups_size":0,"ondisk_log_size":6859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28065792,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":18868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6744","reported_seq":6873,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:27.033286+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:27.033286+0000","last_peered":"2024-09-15T21:20:27.033286+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:27.033286+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6744,"log_dups_size":0,"ondisk_log_size":6744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4169,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3368},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'6943","reported_seq":7065,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:26.985269+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:26.985269+0000","last_peered":"2024-09-15T21:20:26.985269+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:26.985269+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28413952,"num_objects":6937,"num_object_clones":0,"num_object_copies":13874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4579,"num_write_kb":18316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6720","reported_seq":6839,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:26.979719+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:26.979719+0000","last_peered":"2024-09-15T21:20:26.979719+0000","last_clean":"2024-09-15T21:20:26.979719+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:26.979719+0000","last_undegraded":"2024-09-15T21:20:26.979719+0000","last_fullsized":"2024-09-15T21:20:26.979719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6720,"log_dups_size":0,"ondisk_log_size":6720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4505,"num_write_kb":18020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6580","reported_seq":6704,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.909332+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:23.909332+0000","last_peered":"2024-09-15T21:20:23.909332+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:23.909332+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3235}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6673","reported_seq":6795,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:26.965426+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:26.965426+0000","last_peered":"2024-09-15T21:20:26.965426+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:26.965426+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6673,"log_dups_size":0,"ondisk_log_size":6673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27299840,"num_objects":6665,"num_object_clones":0,"num_object_copies":13330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4199,"num_write_kb":16796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3324},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6550","reported_seq":6659,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.874183+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:23.874183+0000","last_peered":"2024-09-15T21:20:23.874183+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:23.874183+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6550,"log_dups_size":0,"ondisk_log_size":6550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26808320,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5101,"num_write_kb":20404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6629","reported_seq":6743,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:23.917803+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:23.917803+0000","last_peered":"2024-09-15T21:20:23.917803+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:23.917803+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6629,"log_dups_size":0,"ondisk_log_size":6629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'6841","reported_seq":6968,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:26.940164+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:26.940164+0000","last_peered":"2024-09-15T21:20:26.940164+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:26.940164+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":6841,"log_dups_size":0,"ondisk_log_size":6841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4148,"num_write_kb":16592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3406},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6681","reported_seq":6810,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:27.010751+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:27.010751+0000","last_peered":"2024-09-15T21:20:27.010751+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:27.010751+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6681,"log_dups_size":0,"ondisk_log_size":6681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27332608,"num_objects":6673,"num_object_clones":0,"num_object_copies":13346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4266,"num_write_kb":17064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6753","reported_seq":6876,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:27.018826+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:27.018826+0000","last_peered":"2024-09-15T21:20:27.018826+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:27.018826+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6753,"log_dups_size":0,"ondisk_log_size":6753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27643904,"num_objects":6749,"num_object_clones":0,"num_object_copies":13498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4400,"num_write_kb":17600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3384},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'6891","reported_seq":7598,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:27.167605+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:27.167605+0000","last_peered":"2024-09-15T21:20:27.167605+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:27.167605+0000","last_undegraded":"2024-09-15T21:20:27.167605+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":6891,"log_dups_size":0,"ondisk_log_size":6891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28177001,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3181,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4551,"num_write_kb":18204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":3181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441373289,"num_objects":107758,"num_object_clones":0,"num_object_copies":215516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3181,"num_objects_unfound":0,"num_objects_dirty":107758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79253,"num_write_kb":317012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3682,"num_bytes_recovered":15077376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":898367488,"data_stored":898283992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107839,"ondisk_log_size":107839,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475080,"kb_used_data":321208,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93896760,"statfs":{"total":96636764160,"available":96150282240,"internally_reserved":0,"allocated":328916992,"data_stored":328651147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157544109},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149738,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581348,"kb_used_data":378964,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93790492,"statfs":{"total":96636764160,"available":96041463808,"internally_reserved":0,"allocated":388059136,"data_stored":387793291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":207220401},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,7,1,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352676,"kb_used_data":176596,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94019164,"statfs":{"total":96636764160,"available":96275623936,"internally_reserved":0,"allocated":180834304,"data_stored":180495400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":388263936,"data_stored":388260457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180781056,"data_stored":180704518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329322496,"data_stored":329319017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:29.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 156 20 109452 109432 2.7394 2.04688 0.0675773 0.0285116 2024-09-15T21:20:30.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 157 20 109985 109965 2.73521 2.08203 0.00826049 0.028552 2024-09-15T21:20:31.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 158 20 110533 110513 2.73145 2.14062 0.0491322 0.0285944 2024-09-15T21:20:31.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:32.514 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 159 20 111068 111048 2.72741 2.08984 0.0337752 0.0286362 2024-09-15T21:20:32.788 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:32.788 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:32.854 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-15T21:20:32.359411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":451871356,"num_objects":110323,"num_object_clones":0,"num_object_copies":220646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":110323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81818,"num_write_kb":327266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3793,"num_bytes_recovered":15527955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110405,"ondisk_log_size":110405,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444052,"kb_used_data":897828,"kb_used_omap":12,"kb_used_meta":546163,"kb_avail":281671468,"statfs":{"total":289910292480,"available":288431583232,"internally_reserved":0,"allocated":919375872,"data_stored":918505278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":559270922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,7,3,11,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7065,"num_write_kb":28260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005601"},"pg_stats":[{"pgid":"2.f","version":"46'6881","reported_seq":6957,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.161396+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:32.161396+0000","last_peered":"2024-09-15T21:20:32.161396+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:32.161396+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6881,"log_dups_size":0,"ondisk_log_size":6881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6868","reported_seq":6964,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:28.881376+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:28.881376+0000","last_peered":"2024-09-15T21:20:28.881376+0000","last_clean":"2024-09-15T21:20:28.881376+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:28.881376+0000","last_undegraded":"2024-09-15T21:20:28.881376+0000","last_fullsized":"2024-09-15T21:20:28.881376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6928","reported_seq":13842,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:31.902695+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:31.902695+0000","last_peered":"2024-09-15T21:20:31.902695+0000","last_clean":"2024-09-15T21:20:31.902695+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:31.902695+0000","last_undegraded":"2024-09-15T21:20:31.902695+0000","last_fullsized":"2024-09-15T21:20:31.902695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6928,"log_dups_size":0,"ondisk_log_size":6928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6902","reported_seq":6971,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:28.964848+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:28.964848+0000","last_peered":"2024-09-15T21:20:28.964848+0000","last_clean":"2024-09-15T21:20:28.964848+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:28.964848+0000","last_undegraded":"2024-09-15T21:20:28.964848+0000","last_fullsized":"2024-09-15T21:20:28.964848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6902,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7025","reported_seq":7143,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.118385+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:32.118385+0000","last_peered":"2024-09-15T21:20:32.118385+0000","last_clean":"2024-09-15T21:20:32.118385+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:32.118385+0000","last_undegraded":"2024-09-15T21:20:32.118385+0000","last_fullsized":"2024-09-15T21:20:32.118385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7025,"log_dups_size":0,"ondisk_log_size":7025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6914","reported_seq":7043,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.210688+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:32.210688+0000","last_peered":"2024-09-15T21:20:32.210688+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:32.210688+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3538},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7111","reported_seq":7230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.134972+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:32.134972+0000","last_peered":"2024-09-15T21:20:32.134972+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:32.134972+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4747,"num_write_kb":18988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6876","reported_seq":6995,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.065735+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:32.065735+0000","last_peered":"2024-09-15T21:20:32.065735+0000","last_clean":"2024-09-15T21:20:32.065735+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:32.065735+0000","last_undegraded":"2024-09-15T21:20:32.065735+0000","last_fullsized":"2024-09-15T21:20:32.065735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6876,"log_dups_size":0,"ondisk_log_size":6876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4661,"num_write_kb":18644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6726","reported_seq":6850,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.934516+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:28.934516+0000","last_peered":"2024-09-15T21:20:28.934516+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:28.934516+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6726,"log_dups_size":0,"ondisk_log_size":6726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5312,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3381}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6839","reported_seq":6961,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.170926+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:32.170926+0000","last_peered":"2024-09-15T21:20:32.170926+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:32.170926+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6839,"log_dups_size":0,"ondisk_log_size":6839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27979776,"num_objects":6831,"num_object_clones":0,"num_object_copies":13662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4365,"num_write_kb":17460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3490},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6699","reported_seq":6808,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.962325+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:28.962325+0000","last_peered":"2024-09-15T21:20:28.962325+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:28.962325+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6699,"log_dups_size":0,"ondisk_log_size":6699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6803","reported_seq":6916,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.970760+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:28.970760+0000","last_peered":"2024-09-15T21:20:28.970760+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:28.970760+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6803,"log_dups_size":0,"ondisk_log_size":6803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27840512,"num_objects":6797,"num_object_clones":0,"num_object_copies":13594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5202,"num_write_kb":20808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7011","reported_seq":7138,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162873+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:32.162873+0000","last_peered":"2024-09-15T21:20:32.162873+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:32.162873+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4318,"num_write_kb":17272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6858","reported_seq":6987,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162887+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:32.162887+0000","last_peered":"2024-09-15T21:20:32.162887+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:32.162887+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28057600,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6908","reported_seq":7031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.098830+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:32.098830+0000","last_peered":"2024-09-15T21:20:32.098830+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:32.098830+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4555,"num_write_kb":18220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7053","reported_seq":7980,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:32.164393+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:32.164393+0000","last_peered":"2024-09-15T21:20:32.164393+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:32.164393+0000","last_undegraded":"2024-09-15T21:20:32.164393+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28840553,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":451871337,"num_objects":110321,"num_object_clones":0,"num_object_copies":220642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":110321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81816,"num_write_kb":327264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3792,"num_bytes_recovered":15527936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":920088576,"data_stored":920005080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110402,"ondisk_log_size":110402,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953511,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484352,"kb_used_data":326320,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93887488,"statfs":{"total":96636764160,"available":96140787712,"internally_reserved":0,"allocated":334151680,"data_stored":333885835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149739,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594604,"kb_used_data":386652,"kb_used_omap":4,"kb_used_meta":207931,"kb_avail":93777236,"statfs":{"total":96636764160,"available":96027889664,"internally_reserved":0,"allocated":395931648,"data_stored":395665803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":212922033},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365096,"kb_used_data":184856,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94006744,"statfs":{"total":96636764160,"available":96262905856,"internally_reserved":0,"allocated":189292544,"data_stored":188953640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396324864,"data_stored":396321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189214720,"data_stored":189138182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334545513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:32.855 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:33.443 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:33.444 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:33.513 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:20:33.512884+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0287036 2024-09-15T21:20:33.514 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:20:33.514 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 160 20 111500 111480 2.72091 1.6875 0.0322694 0.0287036 2024-09-15T21:20:33.518 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-15T21:20:32.359411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":451871356,"num_objects":110323,"num_object_clones":0,"num_object_copies":220646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":110323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81818,"num_write_kb":327266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3793,"num_bytes_recovered":15527955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110405,"ondisk_log_size":110405,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444052,"kb_used_data":897828,"kb_used_omap":12,"kb_used_meta":546163,"kb_avail":281671468,"statfs":{"total":289910292480,"available":288431583232,"internally_reserved":0,"allocated":919375872,"data_stored":918505278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":559270922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,7,3,11,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7065,"num_write_kb":28260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005601"},"pg_stats":[{"pgid":"2.f","version":"46'6881","reported_seq":6957,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.161396+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:32.161396+0000","last_peered":"2024-09-15T21:20:32.161396+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:32.161396+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6881,"log_dups_size":0,"ondisk_log_size":6881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'6868","reported_seq":6964,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:28.881376+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:28.881376+0000","last_peered":"2024-09-15T21:20:28.881376+0000","last_clean":"2024-09-15T21:20:28.881376+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:28.881376+0000","last_undegraded":"2024-09-15T21:20:28.881376+0000","last_fullsized":"2024-09-15T21:20:28.881376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6928","reported_seq":13842,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:31.902695+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:31.902695+0000","last_peered":"2024-09-15T21:20:31.902695+0000","last_clean":"2024-09-15T21:20:31.902695+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:31.902695+0000","last_undegraded":"2024-09-15T21:20:31.902695+0000","last_fullsized":"2024-09-15T21:20:31.902695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6928,"log_dups_size":0,"ondisk_log_size":6928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'6902","reported_seq":6971,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:28.964848+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:28.964848+0000","last_peered":"2024-09-15T21:20:28.964848+0000","last_clean":"2024-09-15T21:20:28.964848+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:28.964848+0000","last_undegraded":"2024-09-15T21:20:28.964848+0000","last_fullsized":"2024-09-15T21:20:28.964848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6902,"num_write_kb":27608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7025","reported_seq":7143,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.118385+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:32.118385+0000","last_peered":"2024-09-15T21:20:32.118385+0000","last_clean":"2024-09-15T21:20:32.118385+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:32.118385+0000","last_undegraded":"2024-09-15T21:20:32.118385+0000","last_fullsized":"2024-09-15T21:20:32.118385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7025,"log_dups_size":0,"ondisk_log_size":7025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6914","reported_seq":7043,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.210688+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:32.210688+0000","last_peered":"2024-09-15T21:20:32.210688+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:32.210688+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3538},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7111","reported_seq":7230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.134972+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:32.134972+0000","last_peered":"2024-09-15T21:20:32.134972+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:32.134972+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4747,"num_write_kb":18988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6876","reported_seq":6995,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.065735+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:32.065735+0000","last_peered":"2024-09-15T21:20:32.065735+0000","last_clean":"2024-09-15T21:20:32.065735+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:32.065735+0000","last_undegraded":"2024-09-15T21:20:32.065735+0000","last_fullsized":"2024-09-15T21:20:32.065735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6876,"log_dups_size":0,"ondisk_log_size":6876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4661,"num_write_kb":18644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6726","reported_seq":6850,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.934516+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:28.934516+0000","last_peered":"2024-09-15T21:20:28.934516+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:28.934516+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6726,"log_dups_size":0,"ondisk_log_size":6726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5312,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3381}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6839","reported_seq":6961,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.170926+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:32.170926+0000","last_peered":"2024-09-15T21:20:32.170926+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:32.170926+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6839,"log_dups_size":0,"ondisk_log_size":6839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27979776,"num_objects":6831,"num_object_clones":0,"num_object_copies":13662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4365,"num_write_kb":17460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3490},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6699","reported_seq":6808,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.962325+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:28.962325+0000","last_peered":"2024-09-15T21:20:28.962325+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:28.962325+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6699,"log_dups_size":0,"ondisk_log_size":6699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5250,"num_write_kb":21000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3428}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6803","reported_seq":6916,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:28.970760+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:28.970760+0000","last_peered":"2024-09-15T21:20:28.970760+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:28.970760+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6803,"log_dups_size":0,"ondisk_log_size":6803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27840512,"num_objects":6797,"num_object_clones":0,"num_object_copies":13594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5202,"num_write_kb":20808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3447}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7011","reported_seq":7138,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162873+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:32.162873+0000","last_peered":"2024-09-15T21:20:32.162873+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:32.162873+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4318,"num_write_kb":17272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6858","reported_seq":6987,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162887+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:32.162887+0000","last_peered":"2024-09-15T21:20:32.162887+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:32.162887+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28057600,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6908","reported_seq":7031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.098830+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:32.098830+0000","last_peered":"2024-09-15T21:20:32.098830+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:32.098830+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4555,"num_write_kb":18220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7053","reported_seq":7980,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:32.164393+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:32.164393+0000","last_peered":"2024-09-15T21:20:32.164393+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:32.164393+0000","last_undegraded":"2024-09-15T21:20:32.164393+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28840553,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":451871337,"num_objects":110321,"num_object_clones":0,"num_object_copies":220642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":110321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81816,"num_write_kb":327264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3792,"num_bytes_recovered":15527936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":920088576,"data_stored":920005080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110402,"ondisk_log_size":110402,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953511,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484352,"kb_used_data":326320,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93887488,"statfs":{"total":96636764160,"available":96140787712,"internally_reserved":0,"allocated":334151680,"data_stored":333885835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,1,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149739,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594604,"kb_used_data":386652,"kb_used_omap":4,"kb_used_meta":207931,"kb_avail":93777236,"statfs":{"total":96636764160,"available":96027889664,"internally_reserved":0,"allocated":395931648,"data_stored":395665803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":212922033},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365096,"kb_used_data":184856,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94006744,"statfs":{"total":96636764160,"available":96262905856,"internally_reserved":0,"allocated":189292544,"data_stored":188953640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396324864,"data_stored":396321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189214720,"data_stored":189138182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334545513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:33.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:33.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'6881", 'reported_seq': 6957, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.161396+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:32.161396+0000', 'last_peered': '2024-09-15T21:20:32.161396+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:32.161396+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 6881, 'log_dups_size': 0, 'ondisk_log_size': 6881, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28184576, 'num_objects': 6881, 'num_object_clones': 0, 'num_object_copies': 13762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6881, 'num_write_kb': 27524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3514}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:33.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'6914", 'reported_seq': 7043, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.210688+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:32.210688+0000', 'last_peered': '2024-09-15T21:20:32.210688+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:32.210688+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 6914, 'log_dups_size': 0, 'ondisk_log_size': 6914, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28299264, 'num_objects': 6909, 'num_object_clones': 0, 'num_object_copies': 13818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4339, 'num_write_kb': 17356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3538}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'7111", 'reported_seq': 7230, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.134972+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:32.134972+0000', 'last_peered': '2024-09-15T21:20:32.134972+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:32.134972+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 7111, 'log_dups_size': 0, 'ondisk_log_size': 7111, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29102080, 'num_objects': 7105, 'num_object_clones': 0, 'num_object_copies': 14210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4747, 'num_write_kb': 18988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3634}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'6726", 'reported_seq': 6850, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:28.934516+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:28.934516+0000', 'last_peered': '2024-09-15T21:20:28.934516+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:28.934516+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 6726, 'log_dups_size': 0, 'ondisk_log_size': 6726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27521024, 'num_objects': 6719, 'num_object_clones': 0, 'num_object_copies': 13438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5312, 'num_write_kb': 21248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 3381}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'6839", 'reported_seq': 6961, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.170926+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:32.170926+0000', 'last_peered': '2024-09-15T21:20:32.170926+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:32.170926+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6839, 'log_dups_size': 0, 'ondisk_log_size': 6839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27979776, 'num_objects': 6831, 'num_object_clones': 0, 'num_object_copies': 13662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4365, 'num_write_kb': 17460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3490}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'6699", 'reported_seq': 6808, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:28.962325+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:28.962325+0000', 'last_peered': '2024-09-15T21:20:28.962325+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:28.962325+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 6699, 'log_dups_size': 0, 'ondisk_log_size': 6699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27418624, 'num_objects': 6694, 'num_object_clones': 0, 'num_object_copies': 13388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5250, 'num_write_kb': 21000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 3428}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:33.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'6803", 'reported_seq': 6916, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:28.970760+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:28.970760+0000', 'last_peered': '2024-09-15T21:20:28.970760+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:28.970760+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6803, 'log_dups_size': 0, 'ondisk_log_size': 6803, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27840512, 'num_objects': 6797, 'num_object_clones': 0, 'num_object_copies': 13594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5202, 'num_write_kb': 20808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 3447}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7011", 'reported_seq': 7138, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.162873+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:32.162873+0000', 'last_peered': '2024-09-15T21:20:32.162873+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:32.162873+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7011, 'log_dups_size': 0, 'ondisk_log_size': 7011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28708864, 'num_objects': 7009, 'num_object_clones': 0, 'num_object_copies': 14018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4318, 'num_write_kb': 17272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3576}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'6858", 'reported_seq': 6987, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.162887+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:32.162887+0000', 'last_peered': '2024-09-15T21:20:32.162887+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:32.162887+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6858, 'log_dups_size': 0, 'ondisk_log_size': 6858, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28057600, 'num_objects': 6850, 'num_object_clones': 0, 'num_object_copies': 13700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4443, 'num_write_kb': 17772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3471}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'6908", 'reported_seq': 7031, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:32.098830+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:32.098830+0000', 'last_peered': '2024-09-15T21:20:32.098830+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:32.098830+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 6908, 'log_dups_size': 0, 'ondisk_log_size': 6908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28278784, 'num_objects': 6904, 'num_object_clones': 0, 'num_object_copies': 13808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4555, 'num_write_kb': 18220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3539}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:33.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7053", 'reported_seq': 7980, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:32.164393+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:32.164393+0000', 'last_peered': '2024-09-15T21:20:32.164393+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:32.164393+0000', 'last_undegraded': '2024-09-15T21:20:32.164393+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7053, 'log_dups_size': 0, 'ondisk_log_size': 7053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28840553, 'num_objects': 7042, 'num_object_clones': 0, 'num_object_copies': 14084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3071, 'num_objects_unfound': 0, 'num_objects_dirty': 7042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4713, 'num_write_kb': 18852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 399, 'num_bytes_recovered': 1634304, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3971}, {'shards': '0,2', 'objects': 3071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:33.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:34.309 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:34.309 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/220646 objects degraded (15.273%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110323,"data_bytes":451871356,"bytes_used":1478709248,"bytes_avail":288431583232,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":220646,"degraded_ratio":0.15272880541682152,"misplaced_objects":3071,"misplaced_total":220646,"misplaced_ratio":0.013918221948279144,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105422,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"write_bytes_sec":2410394,"read_op_per_sec":0,"write_op_per_sec":588},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:34.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:34.330 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:34.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 161 20 111920 111900 2.71419 1.64062 0.0196624 0.0287729 2024-09-15T21:20:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:34.810 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:34.852 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-15T21:20:34.360003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":455377532,"num_objects":111179,"num_object_clones":0,"num_object_copies":222358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":111179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82674,"num_write_kb":330690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3793,"num_bytes_recovered":15527955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111261,"ondisk_log_size":111261,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1471668,"kb_used_data":903236,"kb_used_omap":12,"kb_used_meta":568371,"kb_avail":281643852,"statfs":{"total":289910292480,"available":288403304448,"internally_reserved":0,"allocated":924913664,"data_stored":924043070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":582011914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,4,12,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24715264,"num_objects":6034,"num_object_clones":0,"num_object_copies":12068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-215,"num_objects_unfound":0,"num_objects_dirty":6034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6034,"num_write_kb":24136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":880640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005763"},"pg_stats":[{"pgid":"2.f","version":"46'6881","reported_seq":6957,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.161396+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:32.161396+0000","last_peered":"2024-09-15T21:20:32.161396+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:32.161396+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":6881,"log_dups_size":0,"ondisk_log_size":6881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3514},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7046","reported_seq":7142,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.031271+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:34.031271+0000","last_peered":"2024-09-15T21:20:34.031271+0000","last_clean":"2024-09-15T21:20:34.031271+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:34.031271+0000","last_undegraded":"2024-09-15T21:20:34.031271+0000","last_fullsized":"2024-09-15T21:20:34.031271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":7040,"num_object_clones":0,"num_object_copies":14080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5772,"num_write_kb":23088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'6928","reported_seq":13842,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:31.902695+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:31.902695+0000","last_peered":"2024-09-15T21:20:31.902695+0000","last_clean":"2024-09-15T21:20:31.902695+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:31.902695+0000","last_undegraded":"2024-09-15T21:20:31.902695+0000","last_fullsized":"2024-09-15T21:20:31.902695+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":6928,"log_dups_size":0,"ondisk_log_size":6928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7073","reported_seq":7141,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.033715+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:34.033715+0000","last_peered":"2024-09-15T21:20:34.033715+0000","last_clean":"2024-09-15T21:20:34.033715+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:34.033715+0000","last_undegraded":"2024-09-15T21:20:34.033715+0000","last_fullsized":"2024-09-15T21:20:34.033715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7073,"log_dups_size":0,"ondisk_log_size":7073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7073,"num_write_kb":28292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7025","reported_seq":7143,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.118385+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:32.118385+0000","last_peered":"2024-09-15T21:20:32.118385+0000","last_clean":"2024-09-15T21:20:32.118385+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:32.118385+0000","last_undegraded":"2024-09-15T21:20:32.118385+0000","last_fullsized":"2024-09-15T21:20:32.118385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7025,"log_dups_size":0,"ondisk_log_size":7025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28745728,"num_objects":7018,"num_object_clones":0,"num_object_copies":14036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'6914","reported_seq":7043,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.210688+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:32.210688+0000","last_peered":"2024-09-15T21:20:32.210688+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:32.210688+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3538},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7111","reported_seq":7230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.134972+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:32.134972+0000","last_peered":"2024-09-15T21:20:32.134972+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:32.134972+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4747,"num_write_kb":18988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6876","reported_seq":6995,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:32.065735+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:32.065735+0000","last_peered":"2024-09-15T21:20:32.065735+0000","last_clean":"2024-09-15T21:20:32.065735+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:32.065735+0000","last_undegraded":"2024-09-15T21:20:32.065735+0000","last_fullsized":"2024-09-15T21:20:32.065735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6876,"log_dups_size":0,"ondisk_log_size":6876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28155904,"num_objects":6874,"num_object_clones":0,"num_object_copies":13748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4661,"num_write_kb":18644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427324+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427324+0000","last_peered":"2024-09-15T21:20:16.427324+0000","last_clean":"2024-09-15T21:20:16.427324+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427324+0000","last_undegraded":"2024-09-15T21:20:16.427324+0000","last_fullsized":"2024-09-15T21:20:16.427324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6901","reported_seq":7025,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.790689+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:33.790689+0000","last_peered":"2024-09-15T21:20:33.790689+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:33.790689+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28237824,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5487,"num_write_kb":21948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6839","reported_seq":6961,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.170926+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:32.170926+0000","last_peered":"2024-09-15T21:20:32.170926+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:32.170926+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6839,"log_dups_size":0,"ondisk_log_size":6839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27979776,"num_objects":6831,"num_object_clones":0,"num_object_copies":13662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4365,"num_write_kb":17460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3490},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6858","reported_seq":6967,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.972320+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:33.972320+0000","last_peered":"2024-09-15T21:20:33.972320+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:33.972320+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6976","reported_seq":7088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:34.026109+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:34.026109+0000","last_peered":"2024-09-15T21:20:34.026109+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:34.026109+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":21500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7011","reported_seq":7138,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162873+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:32.162873+0000","last_peered":"2024-09-15T21:20:32.162873+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:32.162873+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4318,"num_write_kb":17272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6858","reported_seq":6987,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.162887+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:32.162887+0000","last_peered":"2024-09-15T21:20:32.162887+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:32.162887+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28057600,"num_objects":6850,"num_object_clones":0,"num_object_copies":13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3471},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'6908","reported_seq":7031,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:32.098830+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:32.098830+0000","last_peered":"2024-09-15T21:20:32.098830+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:32.098830+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":6908,"log_dups_size":0,"ondisk_log_size":6908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4555,"num_write_kb":18220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7053","reported_seq":7980,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:32.164393+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:32.164393+0000","last_peered":"2024-09-15T21:20:32.164393+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:32.164393+0000","last_undegraded":"2024-09-15T21:20:32.164393+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28840553,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":1634304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":455377513,"num_objects":111177,"num_object_clones":0,"num_object_copies":222354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":3071,"num_objects_unfound":0,"num_objects_dirty":111177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82672,"num_write_kb":330688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3792,"num_bytes_recovered":15527936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925601792,"data_stored":925518296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111258,"ondisk_log_size":111258,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511968,"kb_used_data":331728,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93859872,"statfs":{"total":96636764160,"available":96112508928,"internally_reserved":0,"allocated":339689472,"data_stored":339423627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149739,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594604,"kb_used_data":386652,"kb_used_omap":4,"kb_used_meta":207931,"kb_avail":93777236,"statfs":{"total":96636764160,"available":96027889664,"internally_reserved":0,"allocated":395931648,"data_stored":395665803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":212922033},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":365096,"kb_used_data":184856,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94006744,"statfs":{"total":96636764160,"available":96262905856,"internally_reserved":0,"allocated":189292544,"data_stored":188953640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":184544940},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":396324864,"data_stored":396321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189214720,"data_stored":189138182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340062208,"data_stored":340058729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:35.514 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 162 20 112287 112267 2.70628 1.43359 0.0617221 0.0288597 2024-09-15T21:20:36.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 163 20 112824 112804 2.70252 2.09766 0.0647115 0.0288996 2024-09-15T21:20:37.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 164 20 113349 113329 2.69854 2.05078 0.00881373 0.0289423 2024-09-15T21:20:37.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:38.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 165 20 113870 113850 2.69452 2.03516 0.020511 0.0289862 2024-09-15T21:20:38.676 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:38.677 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:38.719 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-15T21:20:38.361831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461861500,"num_objects":112762,"num_object_clones":0,"num_object_copies":225524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":112762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84257,"num_write_kb":337022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3939,"num_bytes_recovered":16125971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112844,"ondisk_log_size":112844,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538996,"kb_used_data":917824,"kb_used_omap":12,"kb_used_meta":621107,"kb_avail":281576524,"statfs":{"total":289910292480,"available":288334360576,"internally_reserved":0,"allocated":939851776,"data_stored":938981182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":636013578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,10,7,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-361,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006621"},"pg_stats":[{"pgid":"2.f","version":"46'7016","reported_seq":7091,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228346+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:37.228346+0000","last_peered":"2024-09-15T21:20:37.228346+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:37.228346+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7016,"log_dups_size":0,"ondisk_log_size":7016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28737536,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":28064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7046","reported_seq":7142,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.031271+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:34.031271+0000","last_peered":"2024-09-15T21:20:34.031271+0000","last_clean":"2024-09-15T21:20:34.031271+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:34.031271+0000","last_undegraded":"2024-09-15T21:20:34.031271+0000","last_fullsized":"2024-09-15T21:20:34.031271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":7040,"num_object_clones":0,"num_object_copies":14080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5772,"num_write_kb":23088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7093","reported_seq":14007,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:36.979872+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:36.979872+0000","last_peered":"2024-09-15T21:20:36.979872+0000","last_clean":"2024-09-15T21:20:36.979872+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:36.979872+0000","last_undegraded":"2024-09-15T21:20:36.979872+0000","last_fullsized":"2024-09-15T21:20:36.979872+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29036544,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7073","reported_seq":7141,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.033715+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:34.033715+0000","last_peered":"2024-09-15T21:20:34.033715+0000","last_clean":"2024-09-15T21:20:34.033715+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:34.033715+0000","last_undegraded":"2024-09-15T21:20:34.033715+0000","last_fullsized":"2024-09-15T21:20:34.033715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7073,"log_dups_size":0,"ondisk_log_size":7073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7073,"num_write_kb":28292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7173","reported_seq":7291,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.218306+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:37.218306+0000","last_peered":"2024-09-15T21:20:37.218306+0000","last_clean":"2024-09-15T21:20:37.218306+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:37.218306+0000","last_undegraded":"2024-09-15T21:20:37.218306+0000","last_fullsized":"2024-09-15T21:20:37.218306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5031,"num_write_kb":20124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7075","reported_seq":7204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.182407+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:37.182407+0000","last_peered":"2024-09-15T21:20:37.182407+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:37.182407+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28958720,"num_objects":7070,"num_object_clones":0,"num_object_copies":14140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7267","reported_seq":7389,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.224223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:37.224223+0000","last_peered":"2024-09-15T21:20:37.224223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:37.224223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7267,"log_dups_size":0,"ondisk_log_size":7267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":19612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6998","reported_seq":7117,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.076695+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:37.076695+0000","last_peered":"2024-09-15T21:20:37.076695+0000","last_clean":"2024-09-15T21:20:37.076695+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:37.076695+0000","last_undegraded":"2024-09-15T21:20:37.076695+0000","last_fullsized":"2024-09-15T21:20:37.076695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28655616,"num_objects":6996,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6901","reported_seq":7025,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.790689+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:33.790689+0000","last_peered":"2024-09-15T21:20:33.790689+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:33.790689+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28237824,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5487,"num_write_kb":21948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6973","reported_seq":7095,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.160853+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:37.160853+0000","last_peered":"2024-09-15T21:20:37.160853+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:37.160853+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4499,"num_write_kb":17996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6858","reported_seq":6967,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.972320+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:33.972320+0000","last_peered":"2024-09-15T21:20:33.972320+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:33.972320+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6976","reported_seq":7088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:34.026109+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:34.026109+0000","last_peered":"2024-09-15T21:20:34.026109+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:34.026109+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":21500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7167","reported_seq":7287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228514+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:37.228514+0000","last_peered":"2024-09-15T21:20:37.228514+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:37.228514+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7167,"log_dups_size":0,"ondisk_log_size":7167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6992","reported_seq":7121,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.179128+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:37.179128+0000","last_peered":"2024-09-15T21:20:37.179128+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:37.179128+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606464,"num_objects":6984,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3605},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7049","reported_seq":7172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.194290+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:37.194290+0000","last_peered":"2024-09-15T21:20:37.194290+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:37.194290+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7049,"log_dups_size":0,"ondisk_log_size":7049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7184","reported_seq":8403,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:37.200552+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:37.200552+0000","last_peered":"2024-09-15T21:20:37.200552+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:37.200552+0000","last_undegraded":"2024-09-15T21:20:37.200552+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29377129,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":19376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":545,"num_bytes_recovered":2232320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4248},{"shards":"0,2","objects":2925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461861481,"num_objects":112760,"num_object_clones":0,"num_object_copies":225520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":112760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84255,"num_write_kb":337020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3938,"num_bytes_recovered":16125952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940335104,"data_stored":940251608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112841,"ondisk_log_size":112841,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511968,"kb_used_data":331728,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93859872,"statfs":{"total":96636764160,"available":96112508928,"internally_reserved":0,"allocated":339689472,"data_stored":339423627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646016,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93725824,"statfs":{"total":96636764160,"available":95975243776,"internally_reserved":0,"allocated":403091456,"data_stored":402825611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":258404017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,5,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381012,"kb_used_data":192452,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93990828,"statfs":{"total":96636764160,"available":96246607872,"internally_reserved":0,"allocated":197070848,"data_stored":196731944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":193064620},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403292160,"data_stored":403288681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196980736,"data_stored":196904198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340062208,"data_stored":340058729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:38.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:39.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 166 20 114384 114364 2.69038 2.00781 0.0296215 0.0290311 2024-09-15T21:20:39.590 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:39.590 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:39.656 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-15T21:20:38.361831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461861500,"num_objects":112762,"num_object_clones":0,"num_object_copies":225524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":112762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84257,"num_write_kb":337022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3939,"num_bytes_recovered":16125971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112844,"ondisk_log_size":112844,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538996,"kb_used_data":917824,"kb_used_omap":12,"kb_used_meta":621107,"kb_avail":281576524,"statfs":{"total":289910292480,"available":288334360576,"internally_reserved":0,"allocated":939851776,"data_stored":938981182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":636013578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,3,10,7,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-361,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006621"},"pg_stats":[{"pgid":"2.f","version":"46'7016","reported_seq":7091,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228346+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:37.228346+0000","last_peered":"2024-09-15T21:20:37.228346+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:37.228346+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7016,"log_dups_size":0,"ondisk_log_size":7016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28737536,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":28064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7046","reported_seq":7142,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.031271+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:34.031271+0000","last_peered":"2024-09-15T21:20:34.031271+0000","last_clean":"2024-09-15T21:20:34.031271+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:34.031271+0000","last_undegraded":"2024-09-15T21:20:34.031271+0000","last_fullsized":"2024-09-15T21:20:34.031271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":7040,"num_object_clones":0,"num_object_copies":14080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5772,"num_write_kb":23088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7093","reported_seq":14007,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:36.979872+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:36.979872+0000","last_peered":"2024-09-15T21:20:36.979872+0000","last_clean":"2024-09-15T21:20:36.979872+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:36.979872+0000","last_undegraded":"2024-09-15T21:20:36.979872+0000","last_fullsized":"2024-09-15T21:20:36.979872+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29036544,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7073","reported_seq":7141,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:34.033715+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:34.033715+0000","last_peered":"2024-09-15T21:20:34.033715+0000","last_clean":"2024-09-15T21:20:34.033715+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:34.033715+0000","last_undegraded":"2024-09-15T21:20:34.033715+0000","last_fullsized":"2024-09-15T21:20:34.033715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7073,"log_dups_size":0,"ondisk_log_size":7073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28971008,"num_objects":7073,"num_object_clones":0,"num_object_copies":14146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7073,"num_write_kb":28292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7173","reported_seq":7291,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.218306+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:37.218306+0000","last_peered":"2024-09-15T21:20:37.218306+0000","last_clean":"2024-09-15T21:20:37.218306+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:37.218306+0000","last_undegraded":"2024-09-15T21:20:37.218306+0000","last_fullsized":"2024-09-15T21:20:37.218306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5031,"num_write_kb":20124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7075","reported_seq":7204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.182407+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:37.182407+0000","last_peered":"2024-09-15T21:20:37.182407+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:37.182407+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28958720,"num_objects":7070,"num_object_clones":0,"num_object_copies":14140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7267","reported_seq":7389,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.224223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:37.224223+0000","last_peered":"2024-09-15T21:20:37.224223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:37.224223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7267,"log_dups_size":0,"ondisk_log_size":7267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":19612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6998","reported_seq":7117,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.076695+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:37.076695+0000","last_peered":"2024-09-15T21:20:37.076695+0000","last_clean":"2024-09-15T21:20:37.076695+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:37.076695+0000","last_undegraded":"2024-09-15T21:20:37.076695+0000","last_fullsized":"2024-09-15T21:20:37.076695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28655616,"num_objects":6996,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'6901","reported_seq":7025,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.790689+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:33.790689+0000","last_peered":"2024-09-15T21:20:33.790689+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:33.790689+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28237824,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5487,"num_write_kb":21948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6973","reported_seq":7095,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.160853+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:37.160853+0000","last_peered":"2024-09-15T21:20:37.160853+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:37.160853+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4499,"num_write_kb":17996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'6858","reported_seq":6967,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:33.972320+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:33.972320+0000","last_peered":"2024-09-15T21:20:33.972320+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:33.972320+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":6858,"log_dups_size":0,"ondisk_log_size":6858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5409,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3587}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'6976","reported_seq":7088,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:34.026109+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:34.026109+0000","last_peered":"2024-09-15T21:20:34.026109+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:34.026109+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":6976,"log_dups_size":0,"ondisk_log_size":6976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":21500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7167","reported_seq":7287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228514+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:37.228514+0000","last_peered":"2024-09-15T21:20:37.228514+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:37.228514+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7167,"log_dups_size":0,"ondisk_log_size":7167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6992","reported_seq":7121,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.179128+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:37.179128+0000","last_peered":"2024-09-15T21:20:37.179128+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:37.179128+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606464,"num_objects":6984,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3605},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7049","reported_seq":7172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.194290+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:37.194290+0000","last_peered":"2024-09-15T21:20:37.194290+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:37.194290+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7049,"log_dups_size":0,"ondisk_log_size":7049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7184","reported_seq":8403,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:37.200552+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:37.200552+0000","last_peered":"2024-09-15T21:20:37.200552+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:37.200552+0000","last_undegraded":"2024-09-15T21:20:37.200552+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29377129,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":19376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":545,"num_bytes_recovered":2232320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4248},{"shards":"0,2","objects":2925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461861481,"num_objects":112760,"num_object_clones":0,"num_object_copies":225520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":112760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84255,"num_write_kb":337020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3938,"num_bytes_recovered":16125952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940335104,"data_stored":940251608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112841,"ondisk_log_size":112841,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511968,"kb_used_data":331728,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93859872,"statfs":{"total":96636764160,"available":96112508928,"internally_reserved":0,"allocated":339689472,"data_stored":339423627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646016,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93725824,"statfs":{"total":96636764160,"available":95975243776,"internally_reserved":0,"allocated":403091456,"data_stored":402825611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":258404017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,5,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381012,"kb_used_data":192452,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93990828,"statfs":{"total":96636764160,"available":96246607872,"internally_reserved":0,"allocated":197070848,"data_stored":196731944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":193064620},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403292160,"data_stored":403288681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196980736,"data_stored":196904198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340062208,"data_stored":340058729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7016", 'reported_seq': 7091, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.228346+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:37.228346+0000', 'last_peered': '2024-09-15T21:20:37.228346+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:37.228346+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7016, 'log_dups_size': 0, 'ondisk_log_size': 7016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28737536, 'num_objects': 7016, 'num_object_clones': 0, 'num_object_copies': 14032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7016, 'num_write_kb': 28064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3649}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7075", 'reported_seq': 7204, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.182407+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:37.182407+0000', 'last_peered': '2024-09-15T21:20:37.182407+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:37.182407+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7075, 'log_dups_size': 0, 'ondisk_log_size': 7075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28958720, 'num_objects': 7070, 'num_object_clones': 0, 'num_object_copies': 14140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4500, 'num_write_kb': 18000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3699}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'7267", 'reported_seq': 7389, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.224223+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:37.224223+0000', 'last_peered': '2024-09-15T21:20:37.224223+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:37.224223+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 7267, 'log_dups_size': 0, 'ondisk_log_size': 7267, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29741056, 'num_objects': 7261, 'num_object_clones': 0, 'num_object_copies': 14522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4903, 'num_write_kb': 19612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3790}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'6901", 'reported_seq': 7025, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:33.790689+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:33.790689+0000', 'last_peered': '2024-09-15T21:20:33.790689+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:33.790689+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 6901, 'log_dups_size': 0, 'ondisk_log_size': 6901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28237824, 'num_objects': 6894, 'num_object_clones': 0, 'num_object_copies': 13788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5487, 'num_write_kb': 21948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 3556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:39.658 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'6973", 'reported_seq': 7095, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.160853+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:37.160853+0000', 'last_peered': '2024-09-15T21:20:37.160853+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:37.160853+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 6973, 'log_dups_size': 0, 'ondisk_log_size': 6973, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28528640, 'num_objects': 6965, 'num_object_clones': 0, 'num_object_copies': 13930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4499, 'num_write_kb': 17996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3624}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'6858", 'reported_seq': 6967, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:33.972320+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:33.972320+0000', 'last_peered': '2024-09-15T21:20:33.972320+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:33.972320+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 6858, 'log_dups_size': 0, 'ondisk_log_size': 6858, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28069888, 'num_objects': 6853, 'num_object_clones': 0, 'num_object_copies': 13706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5409, 'num_write_kb': 21636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 3587}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'6976", 'reported_seq': 7088, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:34.026109+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:34.026109+0000', 'last_peered': '2024-09-15T21:20:34.026109+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:34.026109+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 6976, 'log_dups_size': 0, 'ondisk_log_size': 6976, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28549120, 'num_objects': 6970, 'num_object_clones': 0, 'num_object_copies': 13940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6970, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5375, 'num_write_kb': 21500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 3620}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7167", 'reported_seq': 7287, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.228514+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:37.228514+0000', 'last_peered': '2024-09-15T21:20:37.228514+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:37.228514+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7167, 'log_dups_size': 0, 'ondisk_log_size': 7167, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29347840, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4474, 'num_write_kb': 17896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3732}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'6992", 'reported_seq': 7121, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.179128+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:37.179128+0000', 'last_peered': '2024-09-15T21:20:37.179128+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:37.179128+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 6992, 'log_dups_size': 0, 'ondisk_log_size': 6992, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28606464, 'num_objects': 6984, 'num_object_clones': 0, 'num_object_copies': 13968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4577, 'num_write_kb': 18308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3605}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:39.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7049", 'reported_seq': 7172, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:37.194290+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:37.194290+0000', 'last_peered': '2024-09-15T21:20:37.194290+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:37.194290+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7049, 'log_dups_size': 0, 'ondisk_log_size': 7049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28856320, 'num_objects': 7045, 'num_object_clones': 0, 'num_object_copies': 14090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4696, 'num_write_kb': 18784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3680}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:39.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7184", 'reported_seq': 8403, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:37.200552+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:37.200552+0000', 'last_peered': '2024-09-15T21:20:37.200552+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:37.200552+0000', 'last_undegraded': '2024-09-15T21:20:37.200552+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7184, 'log_dups_size': 0, 'ondisk_log_size': 7184, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29377129, 'num_objects': 7173, 'num_object_clones': 0, 'num_object_copies': 14346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2925, 'num_objects_unfound': 0, 'num_objects_dirty': 7173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4844, 'num_write_kb': 19376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 545, 'num_bytes_recovered': 2232320, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4248}, {'shards': '0,2', 'objects': 2925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:39.660 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:40.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 167 20 114898 114878 2.68629 2.00781 0.0981716 0.0290744 2024-09-15T21:20:40.697 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:40.697 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/226956 objects degraded (14.848%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":253,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113478,"data_bytes":464794236,"bytes_used":1584824320,"bytes_avail":288325468160,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":226956,"degraded_ratio":0.14848252524718447,"misplaced_objects":2925,"misplaced_total":226956,"misplaced_ratio":0.012887960661978533,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87347,"recovering_keys_per_sec":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"write_bytes_sec":1950977,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:40.744 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:40.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:41.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 168 20 115397 115377 2.6819 1.94922 0.0205962 0.0291197 2024-09-15T21:20:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:41.676 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:41.710 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-15T21:20:40.362306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464794236,"num_objects":113478,"num_object_clones":0,"num_object_copies":226956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":113478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84973,"num_write_kb":339886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3939,"num_bytes_recovered":16125971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113560,"ondisk_log_size":113560,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1547680,"kb_used_data":922348,"kb_used_omap":12,"kb_used_meta":625267,"kb_avail":281567840,"statfs":{"total":289910292480,"available":288325468160,"internally_reserved":0,"allocated":944484352,"data_stored":943613758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":640273418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,9,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5718,"num_write_kb":22872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"2.f","version":"46'7016","reported_seq":7091,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228346+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:37.228346+0000","last_peered":"2024-09-15T21:20:37.228346+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:37.228346+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7016,"log_dups_size":0,"ondisk_log_size":7016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28737536,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":28064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7186","reported_seq":7282,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:38.917379+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:38.917379+0000","last_peered":"2024-09-15T21:20:38.917379+0000","last_clean":"2024-09-15T21:20:38.917379+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:38.917379+0000","last_undegraded":"2024-09-15T21:20:38.917379+0000","last_fullsized":"2024-09-15T21:20:38.917379+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29409280,"num_objects":7180,"num_object_clones":0,"num_object_copies":14360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7093","reported_seq":14007,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:36.979872+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:36.979872+0000","last_peered":"2024-09-15T21:20:36.979872+0000","last_clean":"2024-09-15T21:20:36.979872+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:36.979872+0000","last_undegraded":"2024-09-15T21:20:36.979872+0000","last_fullsized":"2024-09-15T21:20:36.979872+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7093,"log_dups_size":0,"ondisk_log_size":7093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29036544,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7185","reported_seq":7252,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:38.939896+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:38.939896+0000","last_peered":"2024-09-15T21:20:38.939896+0000","last_clean":"2024-09-15T21:20:38.939896+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:38.939896+0000","last_undegraded":"2024-09-15T21:20:38.939896+0000","last_fullsized":"2024-09-15T21:20:38.939896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7185,"log_dups_size":0,"ondisk_log_size":7185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29429760,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7185,"num_write_kb":28740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7173","reported_seq":7291,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.218306+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:37.218306+0000","last_peered":"2024-09-15T21:20:37.218306+0000","last_clean":"2024-09-15T21:20:37.218306+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:37.218306+0000","last_undegraded":"2024-09-15T21:20:37.218306+0000","last_fullsized":"2024-09-15T21:20:37.218306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29351936,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5031,"num_write_kb":20124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7075","reported_seq":7204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.182407+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:37.182407+0000","last_peered":"2024-09-15T21:20:37.182407+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:37.182407+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7075,"log_dups_size":0,"ondisk_log_size":7075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28958720,"num_objects":7070,"num_object_clones":0,"num_object_copies":14140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4500,"num_write_kb":18000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7267","reported_seq":7389,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.224223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:37.224223+0000","last_peered":"2024-09-15T21:20:37.224223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:37.224223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7267,"log_dups_size":0,"ondisk_log_size":7267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741056,"num_objects":7261,"num_object_clones":0,"num_object_copies":14522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4903,"num_write_kb":19612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'6998","reported_seq":7117,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.076695+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:37.076695+0000","last_peered":"2024-09-15T21:20:37.076695+0000","last_clean":"2024-09-15T21:20:37.076695+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:37.076695+0000","last_undegraded":"2024-09-15T21:20:37.076695+0000","last_fullsized":"2024-09-15T21:20:37.076695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28655616,"num_objects":6996,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7067","reported_seq":7189,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:39.023190+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:39.023190+0000","last_peered":"2024-09-15T21:20:39.023190+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:39.023190+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7067,"log_dups_size":0,"ondisk_log_size":7067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28917760,"num_objects":7060,"num_object_clones":0,"num_object_copies":14120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5653,"num_write_kb":22612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3722}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'6973","reported_seq":7095,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.160853+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:37.160853+0000","last_peered":"2024-09-15T21:20:37.160853+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:37.160853+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4499,"num_write_kb":17996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3624},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7016","reported_seq":7125,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:38.993726+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:38.993726+0000","last_peered":"2024-09-15T21:20:38.993726+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:38.993726+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7016,"log_dups_size":0,"ondisk_log_size":7016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5567,"num_write_kb":22268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3745}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":78,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.429933+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.429933+0000","last_peered":"2024-09-15T21:20:16.429933+0000","last_clean":"2024-09-15T21:20:16.429933+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.429933+0000","last_undegraded":"2024-09-15T21:20:16.429933+0000","last_fullsized":"2024-09-15T21:20:16.429933+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7116","reported_seq":7230,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:39.007625+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:39.007625+0000","last_peered":"2024-09-15T21:20:39.007625+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:39.007625+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5515,"num_write_kb":22060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3760}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7167","reported_seq":7287,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.228514+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:37.228514+0000","last_peered":"2024-09-15T21:20:37.228514+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:37.228514+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7167,"log_dups_size":0,"ondisk_log_size":7167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431346+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431346+0000","last_peered":"2024-09-15T21:20:16.431346+0000","last_clean":"2024-09-15T21:20:16.431346+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431346+0000","last_undegraded":"2024-09-15T21:20:16.431346+0000","last_fullsized":"2024-09-15T21:20:16.431346+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'6992","reported_seq":7121,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.179128+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:37.179128+0000","last_peered":"2024-09-15T21:20:37.179128+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:37.179128+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606464,"num_objects":6984,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3605},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7049","reported_seq":7172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:37.194290+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:37.194290+0000","last_peered":"2024-09-15T21:20:37.194290+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:37.194290+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7049,"log_dups_size":0,"ondisk_log_size":7049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856320,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7184","reported_seq":8403,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:37.200552+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:37.200552+0000","last_peered":"2024-09-15T21:20:37.200552+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:37.200552+0000","last_undegraded":"2024-09-15T21:20:37.200552+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29377129,"num_objects":7173,"num_object_clones":0,"num_object_copies":14346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":7173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4844,"num_write_kb":19376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":545,"num_bytes_recovered":2232320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4248},{"shards":"0,2","objects":2925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464794217,"num_objects":113476,"num_object_clones":0,"num_object_copies":226952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2925,"num_objects_unfound":0,"num_objects_dirty":113476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84971,"num_write_kb":339884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3938,"num_bytes_recovered":16125952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":945115136,"data_stored":945031640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113557,"ondisk_log_size":113557,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953513,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520652,"kb_used_data":336252,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93851188,"statfs":{"total":96636764160,"available":96103616512,"internally_reserved":0,"allocated":344322048,"data_stored":344056203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149740,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646016,"kb_used_data":393644,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93725824,"statfs":{"total":96636764160,"available":95975243776,"internally_reserved":0,"allocated":403091456,"data_stored":402825611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":258404017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,5,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":381012,"kb_used_data":192452,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93990828,"statfs":{"total":96636764160,"available":96246607872,"internally_reserved":0,"allocated":197070848,"data_stored":196731944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":193064620},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403292160,"data_stored":403288681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196980736,"data_stored":196904198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344842240,"data_stored":344838761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:42.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 169 20 115919 115899 2.6781 2.03906 0.0479662 0.0291639 2024-09-15T21:20:43.517 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 170 20 116457 116437 2.67471 2.10156 0.0502846 0.0292018 2024-09-15T21:20:44.518 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 171 20 116993 116973 2.67131 2.09375 0.0903159 0.0292382 2024-09-15T21:20:44.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:45.389 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:45.389 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:45.406 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-15T21:20:44.363281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475378300,"num_objects":116062,"num_object_clones":0,"num_object_copies":232124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87557,"num_write_kb":350222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":16560147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116144,"ondisk_log_size":116144,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1565828,"kb_used_data":943248,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281549692,"statfs":{"total":289910292480,"available":288306884608,"internally_reserved":0,"allocated":965885952,"data_stored":965015358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":637455370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,11,10,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23506944,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5739,"num_write_kb":22956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003870"},"pg_stats":[{"pgid":"2.f","version":"46'7164","reported_seq":7240,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.127736+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:42.127736+0000","last_peered":"2024-09-15T21:20:42.127736+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:42.127736+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7164,"num_write_kb":28656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7344","reported_seq":7439,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:44.004312+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:44.004312+0000","last_peered":"2024-09-15T21:20:44.004312+0000","last_clean":"2024-09-15T21:20:44.004312+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:44.004312+0000","last_undegraded":"2024-09-15T21:20:44.004312+0000","last_fullsized":"2024-09-15T21:20:44.004312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6070,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7277","reported_seq":14191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.027460+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:42.027460+0000","last_peered":"2024-09-15T21:20:42.027460+0000","last_clean":"2024-09-15T21:20:42.027460+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:42.027460+0000","last_undegraded":"2024-09-15T21:20:42.027460+0000","last_fullsized":"2024-09-15T21:20:42.027460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7277,"log_dups_size":0,"ondisk_log_size":7277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29790208,"num_objects":7273,"num_object_clones":0,"num_object_copies":14546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7358","reported_seq":7426,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:43.999695+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:43.999695+0000","last_peered":"2024-09-15T21:20:43.999695+0000","last_clean":"2024-09-15T21:20:43.999695+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:43.999695+0000","last_undegraded":"2024-09-15T21:20:43.999695+0000","last_fullsized":"2024-09-15T21:20:43.999695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7358,"num_write_kb":29432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7349","reported_seq":7467,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.245204+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:42.245204+0000","last_peered":"2024-09-15T21:20:42.245204+0000","last_clean":"2024-09-15T21:20:42.245204+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:42.245204+0000","last_undegraded":"2024-09-15T21:20:42.245204+0000","last_fullsized":"2024-09-15T21:20:42.245204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30072832,"num_objects":7342,"num_object_clones":0,"num_object_copies":14684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7240","reported_seq":7368,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.217089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:42.217089+0000","last_peered":"2024-09-15T21:20:42.217089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:42.217089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7240,"log_dups_size":0,"ondisk_log_size":7240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7441","reported_seq":7564,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.245306+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:42.245306+0000","last_peered":"2024-09-15T21:20:42.245306+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:42.245306+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":20308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7154","reported_seq":7273,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.215702+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:42.215702+0000","last_peered":"2024-09-15T21:20:42.215702+0000","last_clean":"2024-09-15T21:20:42.215702+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:42.215702+0000","last_undegraded":"2024-09-15T21:20:42.215702+0000","last_fullsized":"2024-09-15T21:20:42.215702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29294592,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":19756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7234","reported_seq":7358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.060056+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:44.060056+0000","last_peered":"2024-09-15T21:20:44.060056+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:44.060056+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7234,"log_dups_size":0,"ondisk_log_size":7234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29601792,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5820,"num_write_kb":23280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7121","reported_seq":7243,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.188847+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:42.188847+0000","last_peered":"2024-09-15T21:20:42.188847+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:42.188847+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7156","reported_seq":7265,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.034906+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:44.034906+0000","last_peered":"2024-09-15T21:20:44.034906+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:44.034906+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7290","reported_seq":7404,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.033103+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:44.033103+0000","last_peered":"2024-09-15T21:20:44.033103+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:44.033103+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29835264,"num_objects":7284,"num_object_clones":0,"num_object_copies":14568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7332","reported_seq":7460,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.250449+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:42.250449+0000","last_peered":"2024-09-15T21:20:42.250449+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:42.250449+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7141","reported_seq":7270,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.224908+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:42.224908+0000","last_peered":"2024-09-15T21:20:42.224908+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:42.224908+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3754},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7203","reported_seq":7326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.253069+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:42.253069+0000","last_peered":"2024-09-15T21:20:42.253069+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:42.253069+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4850,"num_write_kb":19400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3834},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7337","reported_seq":8768,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:42.221872+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:42.221872+0000","last_peered":"2024-09-15T21:20:42.221872+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:42.221872+0000","last_undegraded":"2024-09-15T21:20:42.221872+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30003817,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4507},{"shards":"0,2","objects":2819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475378281,"num_objects":116060,"num_object_clones":0,"num_object_copies":232120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87555,"num_write_kb":350220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4044,"num_bytes_recovered":16560128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966819840,"data_stored":966736344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116141,"ondisk_log_size":116141,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513312,"kb_used_data":341392,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93858528,"statfs":{"total":96636764160,"available":96111132672,"internally_reserved":0,"allocated":349585408,"data_stored":349319563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":176025261},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149741,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675788,"kb_used_data":401208,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93696052,"statfs":{"total":96636764160,"available":95944757248,"internally_reserved":0,"allocated":410836992,"data_stored":410571147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,8,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376728,"kb_used_data":200648,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995112,"statfs":{"total":96636764160,"available":96250994688,"internally_reserved":0,"allocated":205463552,"data_stored":205124648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411090944,"data_stored":411087465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205438976,"data_stored":205362438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350289920,"data_stored":350286441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:45.408 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:45.518 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 172 20 117398 117378 2.66498 1.58203 0.00758608 0.0292651 2024-09-15T21:20:45.791 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:45.791 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:45.841 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-15T21:20:44.363281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475378300,"num_objects":116062,"num_object_clones":0,"num_object_copies":232124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87557,"num_write_kb":350222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":16560147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116144,"ondisk_log_size":116144,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1565828,"kb_used_data":943248,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281549692,"statfs":{"total":289910292480,"available":288306884608,"internally_reserved":0,"allocated":965885952,"data_stored":965015358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":637455370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,11,10,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23506944,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5739,"num_write_kb":22956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003870"},"pg_stats":[{"pgid":"2.f","version":"46'7164","reported_seq":7240,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.127736+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:42.127736+0000","last_peered":"2024-09-15T21:20:42.127736+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:42.127736+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7164,"num_write_kb":28656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7344","reported_seq":7439,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:44.004312+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:44.004312+0000","last_peered":"2024-09-15T21:20:44.004312+0000","last_clean":"2024-09-15T21:20:44.004312+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:44.004312+0000","last_undegraded":"2024-09-15T21:20:44.004312+0000","last_fullsized":"2024-09-15T21:20:44.004312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6070,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7277","reported_seq":14191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.027460+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:42.027460+0000","last_peered":"2024-09-15T21:20:42.027460+0000","last_clean":"2024-09-15T21:20:42.027460+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:42.027460+0000","last_undegraded":"2024-09-15T21:20:42.027460+0000","last_fullsized":"2024-09-15T21:20:42.027460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7277,"log_dups_size":0,"ondisk_log_size":7277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29790208,"num_objects":7273,"num_object_clones":0,"num_object_copies":14546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7358","reported_seq":7426,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:43.999695+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:43.999695+0000","last_peered":"2024-09-15T21:20:43.999695+0000","last_clean":"2024-09-15T21:20:43.999695+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:43.999695+0000","last_undegraded":"2024-09-15T21:20:43.999695+0000","last_fullsized":"2024-09-15T21:20:43.999695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7358,"num_write_kb":29432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7349","reported_seq":7467,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.245204+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:42.245204+0000","last_peered":"2024-09-15T21:20:42.245204+0000","last_clean":"2024-09-15T21:20:42.245204+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:42.245204+0000","last_undegraded":"2024-09-15T21:20:42.245204+0000","last_fullsized":"2024-09-15T21:20:42.245204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30072832,"num_objects":7342,"num_object_clones":0,"num_object_copies":14684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7240","reported_seq":7368,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.217089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:42.217089+0000","last_peered":"2024-09-15T21:20:42.217089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:42.217089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7240,"log_dups_size":0,"ondisk_log_size":7240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7441","reported_seq":7564,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.245306+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:42.245306+0000","last_peered":"2024-09-15T21:20:42.245306+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:42.245306+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":20308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7154","reported_seq":7273,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.215702+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:42.215702+0000","last_peered":"2024-09-15T21:20:42.215702+0000","last_clean":"2024-09-15T21:20:42.215702+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:42.215702+0000","last_undegraded":"2024-09-15T21:20:42.215702+0000","last_fullsized":"2024-09-15T21:20:42.215702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29294592,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":19756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7234","reported_seq":7358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.060056+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:44.060056+0000","last_peered":"2024-09-15T21:20:44.060056+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:44.060056+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7234,"log_dups_size":0,"ondisk_log_size":7234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29601792,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5820,"num_write_kb":23280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7121","reported_seq":7243,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.188847+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:42.188847+0000","last_peered":"2024-09-15T21:20:42.188847+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:42.188847+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7156","reported_seq":7265,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.034906+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:44.034906+0000","last_peered":"2024-09-15T21:20:44.034906+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:44.034906+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7290","reported_seq":7404,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.033103+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:44.033103+0000","last_peered":"2024-09-15T21:20:44.033103+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:44.033103+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29835264,"num_objects":7284,"num_object_clones":0,"num_object_copies":14568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7332","reported_seq":7460,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.250449+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:42.250449+0000","last_peered":"2024-09-15T21:20:42.250449+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:42.250449+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7141","reported_seq":7270,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.224908+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:42.224908+0000","last_peered":"2024-09-15T21:20:42.224908+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:42.224908+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3754},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7203","reported_seq":7326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.253069+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:42.253069+0000","last_peered":"2024-09-15T21:20:42.253069+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:42.253069+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4850,"num_write_kb":19400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3834},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7337","reported_seq":8768,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:42.221872+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:42.221872+0000","last_peered":"2024-09-15T21:20:42.221872+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:42.221872+0000","last_undegraded":"2024-09-15T21:20:42.221872+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30003817,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4507},{"shards":"0,2","objects":2819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475378281,"num_objects":116060,"num_object_clones":0,"num_object_copies":232120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87555,"num_write_kb":350220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4044,"num_bytes_recovered":16560128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966819840,"data_stored":966736344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116141,"ondisk_log_size":116141,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513312,"kb_used_data":341392,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93858528,"statfs":{"total":96636764160,"available":96111132672,"internally_reserved":0,"allocated":349585408,"data_stored":349319563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":176025261},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149741,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675788,"kb_used_data":401208,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93696052,"statfs":{"total":96636764160,"available":95944757248,"internally_reserved":0,"allocated":410836992,"data_stored":410571147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,8,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376728,"kb_used_data":200648,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995112,"statfs":{"total":96636764160,"available":96250994688,"internally_reserved":0,"allocated":205463552,"data_stored":205124648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411090944,"data_stored":411087465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205438976,"data_stored":205362438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350289920,"data_stored":350286441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:45.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:45.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7164", 'reported_seq': 7240, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.127736+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:42.127736+0000', 'last_peered': '2024-09-15T21:20:42.127736+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:42.127736+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7164, 'log_dups_size': 0, 'ondisk_log_size': 7164, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29343744, 'num_objects': 7164, 'num_object_clones': 0, 'num_object_copies': 14328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7164, 'num_write_kb': 28656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3797}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7240", 'reported_seq': 7368, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.217089+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:42.217089+0000', 'last_peered': '2024-09-15T21:20:42.217089+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:42.217089+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7240, 'log_dups_size': 0, 'ondisk_log_size': 7240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29634560, 'num_objects': 7235, 'num_object_clones': 0, 'num_object_copies': 14470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4665, 'num_write_kb': 18660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3864}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'7441", 'reported_seq': 7564, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.245306+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:42.245306+0000', 'last_peered': '2024-09-15T21:20:42.245306+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:42.245306+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 7441, 'log_dups_size': 0, 'ondisk_log_size': 7441, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30453760, 'num_objects': 7435, 'num_object_clones': 0, 'num_object_copies': 14870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5077, 'num_write_kb': 20308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3964}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'7234", 'reported_seq': 7358, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:44.060056+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:44.060056+0000', 'last_peered': '2024-09-15T21:20:44.060056+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:44.060056+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 7234, 'log_dups_size': 0, 'ondisk_log_size': 7234, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29601792, 'num_objects': 7227, 'num_object_clones': 0, 'num_object_copies': 14454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5820, 'num_write_kb': 23280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 3889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:45.846 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'7121", 'reported_seq': 7243, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.188847+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:42.188847+0000', 'last_peered': '2024-09-15T21:20:42.188847+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:42.188847+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 7121, 'log_dups_size': 0, 'ondisk_log_size': 7121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29134848, 'num_objects': 7113, 'num_object_clones': 0, 'num_object_copies': 14226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4647, 'num_write_kb': 18588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3772}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'7156", 'reported_seq': 7265, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:44.034906+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:44.034906+0000', 'last_peered': '2024-09-15T21:20:44.034906+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:44.034906+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 7156, 'log_dups_size': 0, 'ondisk_log_size': 7156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29290496, 'num_objects': 7151, 'num_object_clones': 0, 'num_object_copies': 14302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5707, 'num_write_kb': 22828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 3885}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'7290", 'reported_seq': 7404, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:44.033103+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:44.033103+0000', 'last_peered': '2024-09-15T21:20:44.033103+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:44.033103+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 7290, 'log_dups_size': 0, 'ondisk_log_size': 7290, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29835264, 'num_objects': 7284, 'num_object_clones': 0, 'num_object_copies': 14568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5689, 'num_write_kb': 22756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 3934}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7332", 'reported_seq': 7460, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.250449+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:42.250449+0000', 'last_peered': '2024-09-15T21:20:42.250449+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:42.250449+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7332, 'log_dups_size': 0, 'ondisk_log_size': 7332, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30023680, 'num_objects': 7330, 'num_object_clones': 0, 'num_object_copies': 14660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4639, 'num_write_kb': 18556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3897}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:45.847 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'7141", 'reported_seq': 7270, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.224908+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:42.224908+0000', 'last_peered': '2024-09-15T21:20:42.224908+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:42.224908+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 7141, 'log_dups_size': 0, 'ondisk_log_size': 7141, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29216768, 'num_objects': 7133, 'num_object_clones': 0, 'num_object_copies': 14266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4726, 'num_write_kb': 18904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3754}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:45.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7203", 'reported_seq': 7326, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:42.253069+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:42.253069+0000', 'last_peered': '2024-09-15T21:20:42.253069+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:42.253069+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7203, 'log_dups_size': 0, 'ondisk_log_size': 7203, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29487104, 'num_objects': 7199, 'num_object_clones': 0, 'num_object_copies': 14398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4850, 'num_write_kb': 19400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3834}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:45.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7337", 'reported_seq': 8768, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:42.221872+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:42.221872+0000', 'last_peered': '2024-09-15T21:20:42.221872+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:42.221872+0000', 'last_undegraded': '2024-09-15T21:20:42.221872+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7337, 'log_dups_size': 0, 'ondisk_log_size': 7337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30003817, 'num_objects': 7326, 'num_object_clones': 0, 'num_object_copies': 14652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2819, 'num_objects_unfound': 0, 'num_objects_dirty': 7326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4997, 'num_write_kb': 19988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 651, 'num_bytes_recovered': 2666496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4507}, {'shards': '0,2', 'objects': 2819}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:45.849 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:46.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 173 20 117822 117802 2.65913 1.65625 0.0401394 0.0293719 2024-09-15T21:20:47.081 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:47.094 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/232124 objects degraded (14.518%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116062,"data_bytes":475378300,"bytes_used":1603407872,"bytes_avail":288306884608,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":232124,"degraded_ratio":0.14517671589322947,"misplaced_objects":2819,"misplaced_total":232124,"misplaced_ratio":0.012144371111991868,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85989,"recovering_keys_per_sec":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"write_bytes_sec":1666224,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:47.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:47.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:47.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 174 20 118318 118298 2.65498 1.9375 0.0503765 0.0294174 2024-09-15T21:20:48.084 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:48.085 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:48.172 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-15T21:20:46.363645+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475378300,"num_objects":116062,"num_object_clones":0,"num_object_copies":232124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87557,"num_write_kb":350222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4045,"num_bytes_recovered":16560147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116144,"ondisk_log_size":116144,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1565828,"kb_used_data":943248,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281549692,"statfs":{"total":289910292480,"available":288306884608,"internally_reserved":0,"allocated":965885952,"data_stored":965015358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":637455370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,11,10,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20000768,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4883,"num_write_kb":19532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003641"},"pg_stats":[{"pgid":"2.f","version":"46'7164","reported_seq":7240,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.127736+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:42.127736+0000","last_peered":"2024-09-15T21:20:42.127736+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:42.127736+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29343744,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7164,"num_write_kb":28656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7344","reported_seq":7439,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:44.004312+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:44.004312+0000","last_peered":"2024-09-15T21:20:44.004312+0000","last_clean":"2024-09-15T21:20:44.004312+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:44.004312+0000","last_undegraded":"2024-09-15T21:20:44.004312+0000","last_fullsized":"2024-09-15T21:20:44.004312+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6070,"num_write_kb":24280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7277","reported_seq":14191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.027460+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:42.027460+0000","last_peered":"2024-09-15T21:20:42.027460+0000","last_clean":"2024-09-15T21:20:42.027460+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:42.027460+0000","last_undegraded":"2024-09-15T21:20:42.027460+0000","last_fullsized":"2024-09-15T21:20:42.027460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7277,"log_dups_size":0,"ondisk_log_size":7277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29790208,"num_objects":7273,"num_object_clones":0,"num_object_copies":14546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7358","reported_seq":7426,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:43.999695+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:43.999695+0000","last_peered":"2024-09-15T21:20:43.999695+0000","last_clean":"2024-09-15T21:20:43.999695+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:43.999695+0000","last_undegraded":"2024-09-15T21:20:43.999695+0000","last_fullsized":"2024-09-15T21:20:43.999695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7358,"num_write_kb":29432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7349","reported_seq":7467,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.245204+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:42.245204+0000","last_peered":"2024-09-15T21:20:42.245204+0000","last_clean":"2024-09-15T21:20:42.245204+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:42.245204+0000","last_undegraded":"2024-09-15T21:20:42.245204+0000","last_fullsized":"2024-09-15T21:20:42.245204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30072832,"num_objects":7342,"num_object_clones":0,"num_object_copies":14684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5207,"num_write_kb":20828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7240","reported_seq":7368,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.217089+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:42.217089+0000","last_peered":"2024-09-15T21:20:42.217089+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:42.217089+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7240,"log_dups_size":0,"ondisk_log_size":7240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4665,"num_write_kb":18660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7441","reported_seq":7564,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.245306+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:42.245306+0000","last_peered":"2024-09-15T21:20:42.245306+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:42.245306+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30453760,"num_objects":7435,"num_object_clones":0,"num_object_copies":14870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":20308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3964},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7154","reported_seq":7273,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:42.215702+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:42.215702+0000","last_peered":"2024-09-15T21:20:42.215702+0000","last_clean":"2024-09-15T21:20:42.215702+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:42.215702+0000","last_undegraded":"2024-09-15T21:20:42.215702+0000","last_fullsized":"2024-09-15T21:20:42.215702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29294592,"num_objects":7152,"num_object_clones":0,"num_object_copies":14304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":19756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7234","reported_seq":7358,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.060056+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:44.060056+0000","last_peered":"2024-09-15T21:20:44.060056+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:44.060056+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7234,"log_dups_size":0,"ondisk_log_size":7234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29601792,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5820,"num_write_kb":23280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7121","reported_seq":7243,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.188847+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:42.188847+0000","last_peered":"2024-09-15T21:20:42.188847+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:42.188847+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3772},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7156","reported_seq":7265,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.034906+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:44.034906+0000","last_peered":"2024-09-15T21:20:44.034906+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:44.034906+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29290496,"num_objects":7151,"num_object_clones":0,"num_object_copies":14302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":22828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":3885}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7290","reported_seq":7404,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:44.033103+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:44.033103+0000","last_peered":"2024-09-15T21:20:44.033103+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:44.033103+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29835264,"num_objects":7284,"num_object_clones":0,"num_object_copies":14568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7332","reported_seq":7460,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.250449+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:42.250449+0000","last_peered":"2024-09-15T21:20:42.250449+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:42.250449+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7141","reported_seq":7270,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.224908+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:42.224908+0000","last_peered":"2024-09-15T21:20:42.224908+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:42.224908+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7141,"log_dups_size":0,"ondisk_log_size":7141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3754},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7203","reported_seq":7326,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:42.253069+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:42.253069+0000","last_peered":"2024-09-15T21:20:42.253069+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:42.253069+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4850,"num_write_kb":19400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3834},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430727+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430727+0000","last_peered":"2024-09-15T21:20:16.430727+0000","last_clean":"2024-09-15T21:20:16.430727+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430727+0000","last_undegraded":"2024-09-15T21:20:16.430727+0000","last_fullsized":"2024-09-15T21:20:16.430727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430024+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430024+0000","last_peered":"2024-09-15T21:20:16.430024+0000","last_clean":"2024-09-15T21:20:16.430024+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430024+0000","last_undegraded":"2024-09-15T21:20:16.430024+0000","last_fullsized":"2024-09-15T21:20:16.430024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7337","reported_seq":8768,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:42.221872+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:42.221872+0000","last_peered":"2024-09-15T21:20:42.221872+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:42.221872+0000","last_undegraded":"2024-09-15T21:20:42.221872+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7337,"log_dups_size":0,"ondisk_log_size":7337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30003817,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2666496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4507},{"shards":"0,2","objects":2819}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475378281,"num_objects":116060,"num_object_clones":0,"num_object_copies":232120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2819,"num_objects_unfound":0,"num_objects_dirty":116060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87555,"num_write_kb":350220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4044,"num_bytes_recovered":16560128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966819840,"data_stored":966736344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116141,"ondisk_log_size":116141,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953514,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513312,"kb_used_data":341392,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93858528,"statfs":{"total":96636764160,"available":96111132672,"internally_reserved":0,"allocated":349585408,"data_stored":349319563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":176025261},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149741,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675788,"kb_used_data":401208,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93696052,"statfs":{"total":96636764160,"available":95944757248,"internally_reserved":0,"allocated":410836992,"data_stored":410571147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,8,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376728,"kb_used_data":200648,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995112,"statfs":{"total":96636764160,"available":96250994688,"internally_reserved":0,"allocated":205463552,"data_stored":205124648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":180285100},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411090944,"data_stored":411087465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205438976,"data_stored":205362438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350289920,"data_stored":350286441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:48.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 175 20 118842 118822 2.65151 2.04688 0.0260463 0.0294557 2024-09-15T21:20:49.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 176 20 119365 119345 2.64805 2.04297 0.0665228 0.0294947 2024-09-15T21:20:50.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 177 20 119890 119870 2.64467 2.05078 0.0150602 0.0295326 2024-09-15T21:20:51.175 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:51.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 178 20 120409 120389 2.64121 2.02734 0.0591946 0.0295733 2024-09-15T21:20:52.167 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:52.167 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:52.280 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-15T21:20:50.367733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485347964,"num_objects":118496,"num_object_clones":0,"num_object_copies":236992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2682,"num_objects_unfound":0,"num_objects_dirty":118496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89991,"num_write_kb":359958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4182,"num_bytes_recovered":17121299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118578,"ondisk_log_size":118578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1585436,"kb_used_data":962860,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281530084,"statfs":{"total":289910292480,"available":288286806016,"internally_reserved":0,"allocated":985968640,"data_stored":985098046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":637455370},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,15,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23486464,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5734,"num_write_kb":22936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005901"},"pg_stats":[{"pgid":"2.f","version":"46'7321","reported_seq":7394,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.271033+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:47.271033+0000","last_peered":"2024-09-15T21:20:47.271033+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:47.271033+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7321,"log_dups_size":0,"ondisk_log_size":7321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29986816,"num_objects":7321,"num_object_clones":0,"num_object_copies":14642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7321,"num_write_kb":29284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3954},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7496","reported_seq":7589,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.073906+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:49.073906+0000","last_peered":"2024-09-15T21:20:49.073906+0000","last_clean":"2024-09-15T21:20:49.073906+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:49.073906+0000","last_undegraded":"2024-09-15T21:20:49.073906+0000","last_fullsized":"2024-09-15T21:20:49.073906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7496,"log_dups_size":0,"ondisk_log_size":7496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7435","reported_seq":14349,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:47.076464+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:47.076464+0000","last_peered":"2024-09-15T21:20:47.076464+0000","last_clean":"2024-09-15T21:20:47.076464+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:47.076464+0000","last_undegraded":"2024-09-15T21:20:47.076464+0000","last_fullsized":"2024-09-15T21:20:47.076464+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7435,"log_dups_size":0,"ondisk_log_size":7435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6158,"num_write_kb":24632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7501","reported_seq":7569,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.069261+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:49.069261+0000","last_peered":"2024-09-15T21:20:49.069261+0000","last_clean":"2024-09-15T21:20:49.069261+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:49.069261+0000","last_undegraded":"2024-09-15T21:20:49.069261+0000","last_fullsized":"2024-09-15T21:20:49.069261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7501,"num_write_kb":30004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7512","reported_seq":7630,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:47.236885+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:47.236885+0000","last_peered":"2024-09-15T21:20:47.236885+0000","last_clean":"2024-09-15T21:20:47.236885+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:47.236885+0000","last_undegraded":"2024-09-15T21:20:47.236885+0000","last_fullsized":"2024-09-15T21:20:47.236885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7512,"log_dups_size":0,"ondisk_log_size":7512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5370,"num_write_kb":21480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7389","reported_seq":7518,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.210098+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:47.210098+0000","last_peered":"2024-09-15T21:20:47.210098+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:47.210098+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7389,"log_dups_size":0,"ondisk_log_size":7389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30244864,"num_objects":7384,"num_object_clones":0,"num_object_copies":14768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4013},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7601","reported_seq":7724,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.257705+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:47.257705+0000","last_peered":"2024-09-15T21:20:47.257705+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:47.257705+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7601,"log_dups_size":0,"ondisk_log_size":7601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109120,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5237,"num_write_kb":20948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4124},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7291","reported_seq":7410,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:47.217635+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:47.217635+0000","last_peered":"2024-09-15T21:20:47.217635+0000","last_clean":"2024-09-15T21:20:47.217635+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:47.217635+0000","last_undegraded":"2024-09-15T21:20:47.217635+0000","last_fullsized":"2024-09-15T21:20:47.217635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7291,"log_dups_size":0,"ondisk_log_size":7291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5076,"num_write_kb":20304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7384","reported_seq":7508,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:48.933974+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:48.933974+0000","last_peered":"2024-09-15T21:20:48.933974+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:48.933974+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5970,"num_write_kb":23880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7265","reported_seq":7387,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.195576+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:47.195576+0000","last_peered":"2024-09-15T21:20:47.195576+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:47.195576+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7265,"log_dups_size":0,"ondisk_log_size":7265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29724672,"num_objects":7257,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4791,"num_write_kb":19164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3916},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7305","reported_seq":7414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.060851+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:49.060851+0000","last_peered":"2024-09-15T21:20:49.060851+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:49.060851+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7305,"log_dups_size":0,"ondisk_log_size":7305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7449","reported_seq":7563,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.006838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:49.006838+0000","last_peered":"2024-09-15T21:20:49.006838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:49.006838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5848,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4093}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7466","reported_seq":7592,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.277674+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:47.277674+0000","last_peered":"2024-09-15T21:20:47.277674+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:47.277674+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30572544,"num_objects":7464,"num_object_clones":0,"num_object_copies":14928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4773,"num_write_kb":19092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4031},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7300","reported_seq":7429,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.166788+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:47.166788+0000","last_peered":"2024-09-15T21:20:47.166788+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:47.166788+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7300,"log_dups_size":0,"ondisk_log_size":7300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29868032,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3913},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7368","reported_seq":7490,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:47.234027+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:47.234027+0000","last_peered":"2024-09-15T21:20:47.234027+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:47.234027+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30162944,"num_objects":7364,"num_object_clones":0,"num_object_copies":14728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5015,"num_write_kb":20060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7492","reported_seq":9197,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:47.264964+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:47.264964+0000","last_peered":"2024-09-15T21:20:47.264964+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:47.264964+0000","last_undegraded":"2024-09-15T21:20:47.264964+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7492,"log_dups_size":0,"ondisk_log_size":7492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638697,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2682,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":788,"num_bytes_recovered":3227648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4799},{"shards":"0,2","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485347945,"num_objects":118494,"num_object_clones":0,"num_object_copies":236988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2682,"num_objects_unfound":0,"num_objects_dirty":118494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89989,"num_write_kb":359956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4181,"num_bytes_recovered":17121280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":987189248,"data_stored":987105752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118575,"ondisk_log_size":118575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953515,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522152,"kb_used_data":346072,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93849688,"statfs":{"total":96636764160,"available":96102080512,"internally_reserved":0,"allocated":354377728,"data_stored":354111883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149742,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670376,"kb_used_data":408280,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93701464,"statfs":{"total":96636764160,"available":95950299136,"internally_reserved":0,"allocated":418078720,"data_stored":417812875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":268365489},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,11,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":392908,"kb_used_data":208508,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93978932,"statfs":{"total":96636764160,"available":96234426368,"internally_reserved":0,"allocated":213512192,"data_stored":213173288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":188804780},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":418492416,"data_stored":418488937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213540864,"data_stored":213464326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355155968,"data_stored":355152489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:52.281 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:52.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 179 20 120822 120802 2.63546 1.61328 0.2206 0.0296294 2024-09-15T21:20:52.781 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:52.784 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:52.841 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-15T21:20:52.368160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":492528252,"num_objects":120249,"num_object_clones":0,"num_object_copies":240498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91744,"num_write_kb":366970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":17625107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120331,"ondisk_log_size":120331,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1615428,"kb_used_data":978964,"kb_used_omap":12,"kb_used_meta":636403,"kb_avail":281500092,"statfs":{"total":289910292480,"available":288256094208,"internally_reserved":0,"allocated":1002459136,"data_stored":1001588542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":651676682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,5,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27734016,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-366,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6771,"num_write_kb":27084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":1499136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005853"},"pg_stats":[{"pgid":"2.f","version":"46'7490","reported_seq":7566,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.271185+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:52.271185+0000","last_peered":"2024-09-15T21:20:52.271185+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:52.271185+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7490,"log_dups_size":0,"ondisk_log_size":7490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7490,"num_write_kb":29960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4123},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7496","reported_seq":7589,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.073906+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:49.073906+0000","last_peered":"2024-09-15T21:20:49.073906+0000","last_clean":"2024-09-15T21:20:49.073906+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:49.073906+0000","last_undegraded":"2024-09-15T21:20:49.073906+0000","last_fullsized":"2024-09-15T21:20:49.073906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7496,"log_dups_size":0,"ondisk_log_size":7496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7587","reported_seq":14499,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.156647+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:52.156647+0000","last_peered":"2024-09-15T21:20:52.156647+0000","last_clean":"2024-09-15T21:20:52.156647+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:52.156647+0000","last_undegraded":"2024-09-15T21:20:52.156647+0000","last_fullsized":"2024-09-15T21:20:52.156647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":25240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7501","reported_seq":7569,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.069261+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:49.069261+0000","last_peered":"2024-09-15T21:20:49.069261+0000","last_clean":"2024-09-15T21:20:49.069261+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:49.069261+0000","last_undegraded":"2024-09-15T21:20:49.069261+0000","last_fullsized":"2024-09-15T21:20:49.069261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7501,"num_write_kb":30004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7674","reported_seq":7792,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.240645+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:52.240645+0000","last_peered":"2024-09-15T21:20:52.240645+0000","last_clean":"2024-09-15T21:20:52.240645+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:52.240645+0000","last_undegraded":"2024-09-15T21:20:52.240645+0000","last_fullsized":"2024-09-15T21:20:52.240645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31404032,"num_objects":7667,"num_object_clones":0,"num_object_copies":15334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7545","reported_seq":7674,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.254471+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:52.254471+0000","last_peered":"2024-09-15T21:20:52.254471+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:52.254471+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30883840,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":19880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7766","reported_seq":7889,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.282474+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:52.282474+0000","last_peered":"2024-09-15T21:20:52.282474+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:52.282474+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5402,"num_write_kb":21608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4289},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7456","reported_seq":7575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.236790+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:52.236790+0000","last_peered":"2024-09-15T21:20:52.236790+0000","last_clean":"2024-09-15T21:20:52.236790+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:52.236790+0000","last_undegraded":"2024-09-15T21:20:52.236790+0000","last_fullsized":"2024-09-15T21:20:52.236790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7456,"log_dups_size":0,"ondisk_log_size":7456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5241,"num_write_kb":20964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7384","reported_seq":7508,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:48.933974+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:48.933974+0000","last_peered":"2024-09-15T21:20:48.933974+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:48.933974+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5970,"num_write_kb":23880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7416","reported_seq":7538,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.249066+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:52.249066+0000","last_peered":"2024-09-15T21:20:52.249066+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:52.249066+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30343168,"num_objects":7408,"num_object_clones":0,"num_object_copies":14816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4942,"num_write_kb":19768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7305","reported_seq":7414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.060851+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:49.060851+0000","last_peered":"2024-09-15T21:20:49.060851+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:49.060851+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7305,"log_dups_size":0,"ondisk_log_size":7305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7449","reported_seq":7563,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.006838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:49.006838+0000","last_peered":"2024-09-15T21:20:49.006838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:49.006838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5848,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4093}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7629","reported_seq":7757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.272506+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:52.272506+0000","last_peered":"2024-09-15T21:20:52.272506+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:52.272506+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4194},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7472","reported_seq":7601,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.281679+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:52.281679+0000","last_peered":"2024-09-15T21:20:52.281679+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:52.281679+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7472,"log_dups_size":0,"ondisk_log_size":7472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30572544,"num_objects":7464,"num_object_clones":0,"num_object_copies":14928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5057,"num_write_kb":20228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4085},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7525","reported_seq":7648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.200978+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:52.200978+0000","last_peered":"2024-09-15T21:20:52.200978+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:52.200978+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7525,"log_dups_size":0,"ondisk_log_size":7525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7633","reported_seq":9584,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:52.281074+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:52.281074+0000","last_peered":"2024-09-15T21:20:52.281074+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:52.281074+0000","last_undegraded":"2024-09-15T21:20:52.281074+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31216233,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":911,"num_bytes_recovered":3731456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":492528233,"num_objects":120247,"num_object_clones":0,"num_object_copies":240494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91742,"num_write_kb":366968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4304,"num_bytes_recovered":17625088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003556864,"data_stored":1003473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120328,"ondisk_log_size":120328,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953515,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522152,"kb_used_data":346072,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93849688,"statfs":{"total":96636764160,"available":96102080512,"internally_reserved":0,"allocated":354377728,"data_stored":354111883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686468,"kb_used_data":416052,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93685372,"statfs":{"total":96636764160,"available":95933820928,"internally_reserved":0,"allocated":426037248,"data_stored":425771403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406808,"kb_used_data":216840,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93965032,"statfs":{"total":96636764160,"available":96220192768,"internally_reserved":0,"allocated":222044160,"data_stored":221705256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":194506412},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426414080,"data_stored":426410601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221986816,"data_stored":221910278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355155968,"data_stored":355152489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7490", 'reported_seq': 7566, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.271185+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:52.271185+0000', 'last_peered': '2024-09-15T21:20:52.271185+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:52.271185+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7490, 'log_dups_size': 0, 'ondisk_log_size': 7490, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30679040, 'num_objects': 7490, 'num_object_clones': 0, 'num_object_copies': 14980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7490, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7490, 'num_write_kb': 29960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4123}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7545", 'reported_seq': 7674, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.254471+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:52.254471+0000', 'last_peered': '2024-09-15T21:20:52.254471+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:52.254471+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7545, 'log_dups_size': 0, 'ondisk_log_size': 7545, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30883840, 'num_objects': 7540, 'num_object_clones': 0, 'num_object_copies': 15080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7540, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4970, 'num_write_kb': 19880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4169}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'7766", 'reported_seq': 7889, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.282474+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:52.282474+0000', 'last_peered': '2024-09-15T21:20:52.282474+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:52.282474+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 7766, 'log_dups_size': 0, 'ondisk_log_size': 7766, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31784960, 'num_objects': 7760, 'num_object_clones': 0, 'num_object_copies': 15520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5402, 'num_write_kb': 21608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4289}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'7384", 'reported_seq': 7508, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:48.933974+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:48.933974+0000', 'last_peered': '2024-09-15T21:20:48.933974+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:48.933974+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 7384, 'log_dups_size': 0, 'ondisk_log_size': 7384, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30216192, 'num_objects': 7377, 'num_object_clones': 0, 'num_object_copies': 14754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5970, 'num_write_kb': 23880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:52.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'7416", 'reported_seq': 7538, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.249066+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:52.249066+0000', 'last_peered': '2024-09-15T21:20:52.249066+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:52.249066+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 7416, 'log_dups_size': 0, 'ondisk_log_size': 7416, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30343168, 'num_objects': 7408, 'num_object_clones': 0, 'num_object_copies': 14816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4942, 'num_write_kb': 19768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4067}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'7305", 'reported_seq': 7414, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:49.060851+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:49.060851+0000', 'last_peered': '2024-09-15T21:20:49.060851+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:49.060851+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 7305, 'log_dups_size': 0, 'ondisk_log_size': 7305, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29900800, 'num_objects': 7300, 'num_object_clones': 0, 'num_object_copies': 14600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5856, 'num_write_kb': 23424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 4034}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'7449", 'reported_seq': 7563, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:49.006838+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:49.006838+0000', 'last_peered': '2024-09-15T21:20:49.006838+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:49.006838+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 7449, 'log_dups_size': 0, 'ondisk_log_size': 7449, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30486528, 'num_objects': 7443, 'num_object_clones': 0, 'num_object_copies': 14886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5848, 'num_write_kb': 23392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4093}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7629", 'reported_seq': 7757, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.272506+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:52.272506+0000', 'last_peered': '2024-09-15T21:20:52.272506+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:52.272506+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7629, 'log_dups_size': 0, 'ondisk_log_size': 7629, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31240192, 'num_objects': 7627, 'num_object_clones': 0, 'num_object_copies': 15254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4936, 'num_write_kb': 19744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4194}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'7472", 'reported_seq': 7601, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.281679+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:52.281679+0000', 'last_peered': '2024-09-15T21:20:52.281679+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:52.281679+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 7472, 'log_dups_size': 0, 'ondisk_log_size': 7472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30572544, 'num_objects': 7464, 'num_object_clones': 0, 'num_object_copies': 14928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5057, 'num_write_kb': 20228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4085}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:52.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7525", 'reported_seq': 7648, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:52.200978+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:52.200978+0000', 'last_peered': '2024-09-15T21:20:52.200978+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:52.200978+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7525, 'log_dups_size': 0, 'ondisk_log_size': 7525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30806016, 'num_objects': 7521, 'num_object_clones': 0, 'num_object_copies': 15042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5172, 'num_write_kb': 20688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4156}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:52.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7633", 'reported_seq': 9584, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:52.281074+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:52.281074+0000', 'last_peered': '2024-09-15T21:20:52.281074+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:52.281074+0000', 'last_undegraded': '2024-09-15T21:20:52.281074+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7633, 'log_dups_size': 0, 'ondisk_log_size': 7633, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31216233, 'num_objects': 7622, 'num_object_clones': 0, 'num_object_copies': 15244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2559, 'num_objects_unfound': 0, 'num_objects_dirty': 7622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5293, 'num_write_kb': 21172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 911, 'num_bytes_recovered': 3731456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5063}, {'shards': '0,2', 'objects': 2559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:52.845 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:53.519 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:20:53.519023+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0296927 2024-09-15T21:20:53.520 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:20:53.520 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 180 20 121264 121244 2.63041 1.72656 0.0288644 0.0296927 2024-09-15T21:20:53.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:53.698 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/240498 objects degraded (14.012%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":266,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120249,"data_bytes":492528252,"bytes_used":1654198272,"bytes_avail":288256094208,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":240498,"degraded_ratio":0.14012174737419852,"misplaced_objects":2559,"misplaced_total":240498,"misplaced_ratio":0.010640421126163211,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":124867,"recovering_keys_per_sec":0,"num_objects_recovered":366,"num_bytes_recovered":1499136,"num_keys_recovered":0,"write_bytes_sec":2310041,"read_op_per_sec":0,"write_op_per_sec":563},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:53.750 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:53.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:54.173 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:54.280 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-15T21:20:52.368160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":492528252,"num_objects":120249,"num_object_clones":0,"num_object_copies":240498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91744,"num_write_kb":366970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":17625107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120331,"ondisk_log_size":120331,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1615428,"kb_used_data":978964,"kb_used_omap":12,"kb_used_meta":636403,"kb_avail":281500092,"statfs":{"total":289910292480,"available":288256094208,"internally_reserved":0,"allocated":1002459136,"data_stored":1001588542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":651676682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,5,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27734016,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-366,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6771,"num_write_kb":27084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":1499136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005853"},"pg_stats":[{"pgid":"2.f","version":"46'7490","reported_seq":7566,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.271185+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:52.271185+0000","last_peered":"2024-09-15T21:20:52.271185+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:52.271185+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7490,"log_dups_size":0,"ondisk_log_size":7490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7490,"num_write_kb":29960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4123},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7496","reported_seq":7589,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.073906+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:49.073906+0000","last_peered":"2024-09-15T21:20:49.073906+0000","last_clean":"2024-09-15T21:20:49.073906+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:49.073906+0000","last_undegraded":"2024-09-15T21:20:49.073906+0000","last_fullsized":"2024-09-15T21:20:49.073906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7496,"log_dups_size":0,"ondisk_log_size":7496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6222,"num_write_kb":24888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7587","reported_seq":14499,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.156647+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:52.156647+0000","last_peered":"2024-09-15T21:20:52.156647+0000","last_clean":"2024-09-15T21:20:52.156647+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:52.156647+0000","last_undegraded":"2024-09-15T21:20:52.156647+0000","last_fullsized":"2024-09-15T21:20:52.156647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":25240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7501","reported_seq":7569,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.069261+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:49.069261+0000","last_peered":"2024-09-15T21:20:49.069261+0000","last_clean":"2024-09-15T21:20:49.069261+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:49.069261+0000","last_undegraded":"2024-09-15T21:20:49.069261+0000","last_fullsized":"2024-09-15T21:20:49.069261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7501,"log_dups_size":0,"ondisk_log_size":7501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7501,"num_write_kb":30004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7674","reported_seq":7792,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.240645+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:52.240645+0000","last_peered":"2024-09-15T21:20:52.240645+0000","last_clean":"2024-09-15T21:20:52.240645+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:52.240645+0000","last_undegraded":"2024-09-15T21:20:52.240645+0000","last_fullsized":"2024-09-15T21:20:52.240645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31404032,"num_objects":7667,"num_object_clones":0,"num_object_copies":15334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7545","reported_seq":7674,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.254471+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:52.254471+0000","last_peered":"2024-09-15T21:20:52.254471+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:52.254471+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30883840,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":19880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7766","reported_seq":7889,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.282474+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:52.282474+0000","last_peered":"2024-09-15T21:20:52.282474+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:52.282474+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5402,"num_write_kb":21608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4289},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7456","reported_seq":7575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.236790+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:52.236790+0000","last_peered":"2024-09-15T21:20:52.236790+0000","last_clean":"2024-09-15T21:20:52.236790+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:52.236790+0000","last_undegraded":"2024-09-15T21:20:52.236790+0000","last_fullsized":"2024-09-15T21:20:52.236790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7456,"log_dups_size":0,"ondisk_log_size":7456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5241,"num_write_kb":20964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7384","reported_seq":7508,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:48.933974+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:48.933974+0000","last_peered":"2024-09-15T21:20:48.933974+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:48.933974+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5970,"num_write_kb":23880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7416","reported_seq":7538,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.249066+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:52.249066+0000","last_peered":"2024-09-15T21:20:52.249066+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:52.249066+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30343168,"num_objects":7408,"num_object_clones":0,"num_object_copies":14816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4942,"num_write_kb":19768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7305","reported_seq":7414,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.060851+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:49.060851+0000","last_peered":"2024-09-15T21:20:49.060851+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:49.060851+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7305,"log_dups_size":0,"ondisk_log_size":7305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4034}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7449","reported_seq":7563,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:49.006838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:49.006838+0000","last_peered":"2024-09-15T21:20:49.006838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:49.006838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5848,"num_write_kb":23392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4093}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7629","reported_seq":7757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.272506+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:52.272506+0000","last_peered":"2024-09-15T21:20:52.272506+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:52.272506+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4194},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7472","reported_seq":7601,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.281679+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:52.281679+0000","last_peered":"2024-09-15T21:20:52.281679+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:52.281679+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7472,"log_dups_size":0,"ondisk_log_size":7472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30572544,"num_objects":7464,"num_object_clones":0,"num_object_copies":14928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5057,"num_write_kb":20228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4085},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7525","reported_seq":7648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.200978+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:52.200978+0000","last_peered":"2024-09-15T21:20:52.200978+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:52.200978+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7525,"log_dups_size":0,"ondisk_log_size":7525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7633","reported_seq":9584,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:52.281074+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:52.281074+0000","last_peered":"2024-09-15T21:20:52.281074+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:52.281074+0000","last_undegraded":"2024-09-15T21:20:52.281074+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31216233,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":911,"num_bytes_recovered":3731456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":492528233,"num_objects":120247,"num_object_clones":0,"num_object_copies":240494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91742,"num_write_kb":366968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4304,"num_bytes_recovered":17625088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1003556864,"data_stored":1003473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120328,"ondisk_log_size":120328,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953515,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522152,"kb_used_data":346072,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93849688,"statfs":{"total":96636764160,"available":96102080512,"internally_reserved":0,"allocated":354377728,"data_stored":354111883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686468,"kb_used_data":416052,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93685372,"statfs":{"total":96636764160,"available":95933820928,"internally_reserved":0,"allocated":426037248,"data_stored":425771403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406808,"kb_used_data":216840,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93965032,"statfs":{"total":96636764160,"available":96220192768,"internally_reserved":0,"allocated":222044160,"data_stored":221705256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":194506412},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426414080,"data_stored":426410601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221986816,"data_stored":221910278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355155968,"data_stored":355152489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:54.523 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 181 20 121669 121649 2.62456 1.58203 0.0437963 0.0297596 2024-09-15T21:20:55.524 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 182 20 122200 122180 2.62154 2.07422 0.0330643 0.0297932 2024-09-15T21:20:56.526 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 183 20 122616 122596 2.61606 1.625 0.0219017 0.0298562 2024-09-15T21:20:57.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:57.530 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 184 20 123121 123101 2.61251 1.97266 0.0318778 0.0298939 2024-09-15T21:20:58.114 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:58.115 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:58.157 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-15T21:20:56.369054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495547004,"num_objects":120986,"num_object_clones":0,"num_object_copies":241972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92481,"num_write_kb":369918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4305,"num_bytes_recovered":17625107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121068,"ondisk_log_size":121068,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1624844,"kb_used_data":984220,"kb_used_omap":12,"kb_used_meta":640563,"kb_avail":281490676,"statfs":{"total":289910292480,"available":288246452224,"internally_reserved":0,"allocated":1007841280,"data_stored":1006970686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":655936522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,3,6,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4924,"num_write_kb":19696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005772"},"pg_stats":[{"pgid":"2.f","version":"46'7490","reported_seq":7566,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.271185+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:52.271185+0000","last_peered":"2024-09-15T21:20:52.271185+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:52.271185+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7490,"log_dups_size":0,"ondisk_log_size":7490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679040,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7490,"num_write_kb":29960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4123},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7633","reported_seq":7729,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.077123+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:54.077123+0000","last_peered":"2024-09-15T21:20:54.077123+0000","last_clean":"2024-09-15T21:20:54.077123+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:54.077123+0000","last_undegraded":"2024-09-15T21:20:54.077123+0000","last_fullsized":"2024-09-15T21:20:54.077123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7587","reported_seq":14499,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.156647+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:52.156647+0000","last_peered":"2024-09-15T21:20:52.156647+0000","last_clean":"2024-09-15T21:20:52.156647+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:52.156647+0000","last_undegraded":"2024-09-15T21:20:52.156647+0000","last_fullsized":"2024-09-15T21:20:52.156647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31059968,"num_objects":7583,"num_object_clones":0,"num_object_copies":15166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":25240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7648","reported_seq":7717,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.092600+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:54.092600+0000","last_peered":"2024-09-15T21:20:54.092600+0000","last_clean":"2024-09-15T21:20:54.092600+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:54.092600+0000","last_undegraded":"2024-09-15T21:20:54.092600+0000","last_fullsized":"2024-09-15T21:20:54.092600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7648,"log_dups_size":0,"ondisk_log_size":7648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31326208,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7648,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7674","reported_seq":7792,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.240645+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:52.240645+0000","last_peered":"2024-09-15T21:20:52.240645+0000","last_clean":"2024-09-15T21:20:52.240645+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:52.240645+0000","last_undegraded":"2024-09-15T21:20:52.240645+0000","last_fullsized":"2024-09-15T21:20:52.240645+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31404032,"num_objects":7667,"num_object_clones":0,"num_object_copies":15334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7545","reported_seq":7674,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.254471+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:52.254471+0000","last_peered":"2024-09-15T21:20:52.254471+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:52.254471+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30883840,"num_objects":7540,"num_object_clones":0,"num_object_copies":15080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":19880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7766","reported_seq":7889,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.282474+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:52.282474+0000","last_peered":"2024-09-15T21:20:52.282474+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:52.282474+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7766,"log_dups_size":0,"ondisk_log_size":7766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5402,"num_write_kb":21608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4289},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7456","reported_seq":7575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.236790+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:52.236790+0000","last_peered":"2024-09-15T21:20:52.236790+0000","last_clean":"2024-09-15T21:20:52.236790+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:52.236790+0000","last_undegraded":"2024-09-15T21:20:52.236790+0000","last_fullsized":"2024-09-15T21:20:52.236790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7456,"log_dups_size":0,"ondisk_log_size":7456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30531584,"num_objects":7454,"num_object_clones":0,"num_object_copies":14908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5241,"num_write_kb":20964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7533","reported_seq":7656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:54.056686+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:54.056686+0000","last_peered":"2024-09-15T21:20:54.056686+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:54.056686+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30826496,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6119,"num_write_kb":24476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7416","reported_seq":7538,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.249066+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:52.249066+0000","last_peered":"2024-09-15T21:20:52.249066+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:52.249066+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30343168,"num_objects":7408,"num_object_clones":0,"num_object_copies":14816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4942,"num_write_kb":19768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4067},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7463","reported_seq":7568,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.999921+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:53.999921+0000","last_peered":"2024-09-15T21:20:53.999921+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:53.999921+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7595","reported_seq":7709,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.989182+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:53.989182+0000","last_peered":"2024-09-15T21:20:53.989182+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:53.989182+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7595,"log_dups_size":0,"ondisk_log_size":7595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5994,"num_write_kb":23976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7629","reported_seq":7757,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.272506+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:52.272506+0000","last_peered":"2024-09-15T21:20:52.272506+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:52.272506+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":19744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4194},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7472","reported_seq":7601,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.281679+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:52.281679+0000","last_peered":"2024-09-15T21:20:52.281679+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:52.281679+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7472,"log_dups_size":0,"ondisk_log_size":7472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30572544,"num_objects":7464,"num_object_clones":0,"num_object_copies":14928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5057,"num_write_kb":20228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4085},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7525","reported_seq":7648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:52.200978+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:52.200978+0000","last_peered":"2024-09-15T21:20:52.200978+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:52.200978+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7525,"log_dups_size":0,"ondisk_log_size":7525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":20688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7633","reported_seq":9584,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:52.281074+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:52.281074+0000","last_peered":"2024-09-15T21:20:52.281074+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:52.281074+0000","last_undegraded":"2024-09-15T21:20:52.281074+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31216233,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":911,"num_bytes_recovered":3731456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495546985,"num_objects":120984,"num_object_clones":0,"num_object_copies":241968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2559,"num_objects_unfound":0,"num_objects_dirty":120984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92479,"num_write_kb":369916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4304,"num_bytes_recovered":17625088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008381952,"data_stored":1008298456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121065,"ondisk_log_size":121065,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531568,"kb_used_data":351328,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93840272,"statfs":{"total":96636764160,"available":96092438528,"internally_reserved":0,"allocated":359759872,"data_stored":359494027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686468,"kb_used_data":416052,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93685372,"statfs":{"total":96636764160,"available":95933820928,"internally_reserved":0,"allocated":426037248,"data_stored":425771403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406808,"kb_used_data":216840,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93965032,"statfs":{"total":96636764160,"available":96220192768,"internally_reserved":0,"allocated":222044160,"data_stored":221705256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":194506412},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426414080,"data_stored":426410601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221986816,"data_stored":221910278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359981056,"data_stored":359977577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:58.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:58.530 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 185 20 123594 123574 2.60838 1.84766 0.0322452 0.0299426 2024-09-15T21:20:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:58.845 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:20:58.913 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-15T21:20:58.369598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":501940860,"num_objects":122547,"num_object_clones":0,"num_object_copies":245094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":122547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94042,"num_write_kb":376162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4485,"num_bytes_recovered":18362387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122629,"ondisk_log_size":122629,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1651148,"kb_used_data":998044,"kb_used_omap":12,"kb_used_meta":653043,"kb_avail":281464372,"statfs":{"total":289910292480,"available":288219516928,"internally_reserved":0,"allocated":1021997056,"data_stored":1021126462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":668716042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,9,7,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005953"},"pg_stats":[{"pgid":"2.f","version":"46'7632","reported_seq":7708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.250417+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:57.250417+0000","last_peered":"2024-09-15T21:20:57.250417+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:57.250417+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7632,"log_dups_size":0,"ondisk_log_size":7632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":7632,"num_object_clones":0,"num_object_copies":15264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":30528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4265},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7633","reported_seq":7729,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.077123+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:54.077123+0000","last_peered":"2024-09-15T21:20:54.077123+0000","last_clean":"2024-09-15T21:20:54.077123+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:54.077123+0000","last_undegraded":"2024-09-15T21:20:54.077123+0000","last_fullsized":"2024-09-15T21:20:54.077123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7753","reported_seq":14668,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.201843+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:57.201843+0000","last_peered":"2024-09-15T21:20:57.201843+0000","last_clean":"2024-09-15T21:20:57.201843+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:57.201843+0000","last_undegraded":"2024-09-15T21:20:57.201843+0000","last_fullsized":"2024-09-15T21:20:57.201843+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6476,"num_write_kb":25904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7648","reported_seq":7717,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.092600+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:54.092600+0000","last_peered":"2024-09-15T21:20:54.092600+0000","last_clean":"2024-09-15T21:20:54.092600+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:54.092600+0000","last_undegraded":"2024-09-15T21:20:54.092600+0000","last_fullsized":"2024-09-15T21:20:54.092600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7648,"log_dups_size":0,"ondisk_log_size":7648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31326208,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7648,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7817","reported_seq":7935,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.271887+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:57.271887+0000","last_peered":"2024-09-15T21:20:57.271887+0000","last_clean":"2024-09-15T21:20:57.271887+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:57.271887+0000","last_undegraded":"2024-09-15T21:20:57.271887+0000","last_fullsized":"2024-09-15T21:20:57.271887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7679","reported_seq":7808,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.304694+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:57.304694+0000","last_peered":"2024-09-15T21:20:57.304694+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:57.304694+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7679,"log_dups_size":0,"ondisk_log_size":7679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31432704,"num_objects":7674,"num_object_clones":0,"num_object_copies":15348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5104,"num_write_kb":20416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4303},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7903","reported_seq":8022,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.237010+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:57.237010+0000","last_peered":"2024-09-15T21:20:57.237010+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:57.237010+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5539,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7609","reported_seq":7728,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.250469+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:57.250469+0000","last_peered":"2024-09-15T21:20:57.250469+0000","last_clean":"2024-09-15T21:20:57.250469+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:57.250469+0000","last_undegraded":"2024-09-15T21:20:57.250469+0000","last_fullsized":"2024-09-15T21:20:57.250469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31158272,"num_objects":7607,"num_object_clones":0,"num_object_copies":15214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5394,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7533","reported_seq":7656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:54.056686+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:54.056686+0000","last_peered":"2024-09-15T21:20:54.056686+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:54.056686+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30826496,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6119,"num_write_kb":24476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7541","reported_seq":7663,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.274840+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:57.274840+0000","last_peered":"2024-09-15T21:20:57.274840+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:57.274840+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30855168,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5067,"num_write_kb":20268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4192},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7463","reported_seq":7568,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.999921+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:53.999921+0000","last_peered":"2024-09-15T21:20:53.999921+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:53.999921+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7595","reported_seq":7709,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.989182+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:53.989182+0000","last_peered":"2024-09-15T21:20:53.989182+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:53.989182+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7595,"log_dups_size":0,"ondisk_log_size":7595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5994,"num_write_kb":23976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7775","reported_seq":7902,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.325116+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:57.325116+0000","last_peered":"2024-09-15T21:20:57.325116+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:57.325116+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7775,"log_dups_size":0,"ondisk_log_size":7775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31838208,"num_objects":7773,"num_object_clones":0,"num_object_copies":15546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7624","reported_seq":7753,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.282687+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:57.282687+0000","last_peered":"2024-09-15T21:20:57.282687+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:57.282687+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7624,"log_dups_size":0,"ondisk_log_size":7624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5209,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7661","reported_seq":7784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.309759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:57.309759+0000","last_peered":"2024-09-15T21:20:57.309759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:57.309759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31363072,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7760","reported_seq":10071,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:57.312213+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:57.312213+0000","last_peered":"2024-09-15T21:20:57.312213+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:57.312213+0000","last_undegraded":"2024-09-15T21:20:57.312213+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7760,"log_dups_size":0,"ondisk_log_size":7760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31736425,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5420,"num_write_kb":21680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5370},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":501940841,"num_objects":122545,"num_object_clones":0,"num_object_copies":245090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":122545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94040,"num_write_kb":376160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4484,"num_bytes_recovered":18362368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1022812160,"data_stored":1022728664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122626,"ondisk_log_size":122626,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531568,"kb_used_data":351328,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93840272,"statfs":{"total":96636764160,"available":96092438528,"internally_reserved":0,"allocated":359759872,"data_stored":359494027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697172,"kb_used_data":422596,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674668,"statfs":{"total":96636764160,"available":95922860032,"internally_reserved":0,"allocated":432738304,"data_stored":432472459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422408,"kb_used_data":224120,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93949432,"statfs":{"total":96636764160,"available":96204218368,"internally_reserved":0,"allocated":229498880,"data_stored":229159976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":203026092},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433143808,"data_stored":433140329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229687296,"data_stored":229610758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359981056,"data_stored":359977577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:58.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:20:58.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7632", 'reported_seq': 7708, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.250417+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:20:57.250417+0000', 'last_peered': '2024-09-15T21:20:57.250417+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:20:57.250417+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7632, 'log_dups_size': 0, 'ondisk_log_size': 7632, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31260672, 'num_objects': 7632, 'num_object_clones': 0, 'num_object_copies': 15264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7632, 'num_write_kb': 30528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4265}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7679", 'reported_seq': 7808, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.304694+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:20:57.304694+0000', 'last_peered': '2024-09-15T21:20:57.304694+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:20:57.304694+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7679, 'log_dups_size': 0, 'ondisk_log_size': 7679, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31432704, 'num_objects': 7674, 'num_object_clones': 0, 'num_object_copies': 15348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7674, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5104, 'num_write_kb': 20416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4303}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'7903", 'reported_seq': 8022, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.237010+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:20:57.237010+0000', 'last_peered': '2024-09-15T21:20:57.237010+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:20:57.237010+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 7903, 'log_dups_size': 0, 'ondisk_log_size': 7903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32346112, 'num_objects': 7897, 'num_object_clones': 0, 'num_object_copies': 15794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5539, 'num_write_kb': 22156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4426}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'7533", 'reported_seq': 7656, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:54.056686+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:20:54.056686+0000', 'last_peered': '2024-09-15T21:20:54.056686+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:20:54.056686+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 7533, 'log_dups_size': 0, 'ondisk_log_size': 7533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30826496, 'num_objects': 7526, 'num_object_clones': 0, 'num_object_copies': 15052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6119, 'num_write_kb': 24476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 4188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:58.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'7541", 'reported_seq': 7663, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.274840+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:20:57.274840+0000', 'last_peered': '2024-09-15T21:20:57.274840+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:20:57.274840+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 7541, 'log_dups_size': 0, 'ondisk_log_size': 7541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30855168, 'num_objects': 7533, 'num_object_clones': 0, 'num_object_copies': 15066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5067, 'num_write_kb': 20268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4192}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'7463", 'reported_seq': 7568, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:53.999921+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:20:53.999921+0000', 'last_peered': '2024-09-15T21:20:53.999921+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:20:53.999921+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 7463, 'log_dups_size': 0, 'ondisk_log_size': 7463, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30547968, 'num_objects': 7458, 'num_object_clones': 0, 'num_object_copies': 14916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6014, 'num_write_kb': 24056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'7595", 'reported_seq': 7709, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:53.989182+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:20:53.989182+0000', 'last_peered': '2024-09-15T21:20:53.989182+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:20:53.989182+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 7595, 'log_dups_size': 0, 'ondisk_log_size': 7595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31084544, 'num_objects': 7589, 'num_object_clones': 0, 'num_object_copies': 15178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5994, 'num_write_kb': 23976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4239}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7775", 'reported_seq': 7902, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.325116+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:20:57.325116+0000', 'last_peered': '2024-09-15T21:20:57.325116+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:20:57.325116+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7775, 'log_dups_size': 0, 'ondisk_log_size': 7775, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31838208, 'num_objects': 7773, 'num_object_clones': 0, 'num_object_copies': 15546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5082, 'num_write_kb': 20328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4340}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'7624", 'reported_seq': 7753, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.282687+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:20:57.282687+0000', 'last_peered': '2024-09-15T21:20:57.282687+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:20:57.282687+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 7624, 'log_dups_size': 0, 'ondisk_log_size': 7624, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31195136, 'num_objects': 7616, 'num_object_clones': 0, 'num_object_copies': 15232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5209, 'num_write_kb': 20836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4237}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:20:58.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7661", 'reported_seq': 7784, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:20:57.309759+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:20:57.309759+0000', 'last_peered': '2024-09-15T21:20:57.309759+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:20:57.309759+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7661, 'log_dups_size': 0, 'ondisk_log_size': 7661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31363072, 'num_objects': 7657, 'num_object_clones': 0, 'num_object_copies': 15314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5308, 'num_write_kb': 21232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4292}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:20:58.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7760", 'reported_seq': 10071, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:20:57.312213+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:20:57.312213+0000', 'last_peered': '2024-09-15T21:20:57.312213+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:20:57.312213+0000', 'last_undegraded': '2024-09-15T21:20:57.312213+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7760, 'log_dups_size': 0, 'ondisk_log_size': 7760, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31736425, 'num_objects': 7749, 'num_object_clones': 0, 'num_object_copies': 15498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2379, 'num_objects_unfound': 0, 'num_objects_dirty': 7749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5420, 'num_write_kb': 21680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1091, 'num_bytes_recovered': 4468736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5370}, {'shards': '0,2', 'objects': 2379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:20:58.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:59.530 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 186 20 124022 124002 2.60334 1.67188 0.00770883 0.0299809 2024-09-15T21:20:59.666 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:20:59.666 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/245094 objects degraded (13.749%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":272,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122547,"data_bytes":501940860,"bytes_used":1690775552,"bytes_avail":288219516928,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":245094,"degraded_ratio":0.13749418590418369,"misplaced_objects":2379,"misplaced_total":245094,"misplaced_ratio":0.0097064799627897876,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":150112,"recovering_keys_per_sec":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"write_bytes_sec":2212449,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:59.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:20:59.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:00.343 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-15T21:20:58.369598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":501940860,"num_objects":122547,"num_object_clones":0,"num_object_copies":245094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":122547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94042,"num_write_kb":376162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4485,"num_bytes_recovered":18362387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122629,"ondisk_log_size":122629,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1651148,"kb_used_data":998044,"kb_used_omap":12,"kb_used_meta":653043,"kb_avail":281464372,"statfs":{"total":289910292480,"available":288219516928,"internally_reserved":0,"allocated":1021997056,"data_stored":1021126462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":668716042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,9,7,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-440,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005953"},"pg_stats":[{"pgid":"2.f","version":"46'7632","reported_seq":7708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.250417+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:57.250417+0000","last_peered":"2024-09-15T21:20:57.250417+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:57.250417+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7632,"log_dups_size":0,"ondisk_log_size":7632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":7632,"num_object_clones":0,"num_object_copies":15264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":30528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4265},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7633","reported_seq":7729,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.077123+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:54.077123+0000","last_peered":"2024-09-15T21:20:54.077123+0000","last_clean":"2024-09-15T21:20:54.077123+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:54.077123+0000","last_undegraded":"2024-09-15T21:20:54.077123+0000","last_fullsized":"2024-09-15T21:20:54.077123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7633,"log_dups_size":0,"ondisk_log_size":7633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240192,"num_objects":7627,"num_object_clones":0,"num_object_copies":15254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7753","reported_seq":14668,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.201843+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:20:57.201843+0000","last_peered":"2024-09-15T21:20:57.201843+0000","last_clean":"2024-09-15T21:20:57.201843+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:20:57.201843+0000","last_undegraded":"2024-09-15T21:20:57.201843+0000","last_fullsized":"2024-09-15T21:20:57.201843+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6476,"num_write_kb":25904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7648","reported_seq":7717,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.092600+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:54.092600+0000","last_peered":"2024-09-15T21:20:54.092600+0000","last_clean":"2024-09-15T21:20:54.092600+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:54.092600+0000","last_undegraded":"2024-09-15T21:20:54.092600+0000","last_fullsized":"2024-09-15T21:20:54.092600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7648,"log_dups_size":0,"ondisk_log_size":7648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31326208,"num_objects":7648,"num_object_clones":0,"num_object_copies":15296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7648,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7817","reported_seq":7935,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.271887+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:57.271887+0000","last_peered":"2024-09-15T21:20:57.271887+0000","last_clean":"2024-09-15T21:20:57.271887+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:57.271887+0000","last_undegraded":"2024-09-15T21:20:57.271887+0000","last_fullsized":"2024-09-15T21:20:57.271887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7679","reported_seq":7808,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.304694+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:57.304694+0000","last_peered":"2024-09-15T21:20:57.304694+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:57.304694+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7679,"log_dups_size":0,"ondisk_log_size":7679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31432704,"num_objects":7674,"num_object_clones":0,"num_object_copies":15348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5104,"num_write_kb":20416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4303},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7903","reported_seq":8022,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.237010+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:57.237010+0000","last_peered":"2024-09-15T21:20:57.237010+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:57.237010+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5539,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7609","reported_seq":7728,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.250469+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:57.250469+0000","last_peered":"2024-09-15T21:20:57.250469+0000","last_clean":"2024-09-15T21:20:57.250469+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:57.250469+0000","last_undegraded":"2024-09-15T21:20:57.250469+0000","last_fullsized":"2024-09-15T21:20:57.250469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31158272,"num_objects":7607,"num_object_clones":0,"num_object_copies":15214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5394,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7533","reported_seq":7656,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:54.056686+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:54.056686+0000","last_peered":"2024-09-15T21:20:54.056686+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:54.056686+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7533,"log_dups_size":0,"ondisk_log_size":7533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30826496,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6119,"num_write_kb":24476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7541","reported_seq":7663,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.274840+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:57.274840+0000","last_peered":"2024-09-15T21:20:57.274840+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:57.274840+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30855168,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5067,"num_write_kb":20268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4192},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7463","reported_seq":7568,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.999921+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:53.999921+0000","last_peered":"2024-09-15T21:20:53.999921+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:53.999921+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7595","reported_seq":7709,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:53.989182+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:53.989182+0000","last_peered":"2024-09-15T21:20:53.989182+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:53.989182+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7595,"log_dups_size":0,"ondisk_log_size":7595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31084544,"num_objects":7589,"num_object_clones":0,"num_object_copies":15178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5994,"num_write_kb":23976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4239}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7775","reported_seq":7902,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.325116+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:57.325116+0000","last_peered":"2024-09-15T21:20:57.325116+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:57.325116+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7775,"log_dups_size":0,"ondisk_log_size":7775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31838208,"num_objects":7773,"num_object_clones":0,"num_object_copies":15546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7624","reported_seq":7753,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.282687+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:57.282687+0000","last_peered":"2024-09-15T21:20:57.282687+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:57.282687+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7624,"log_dups_size":0,"ondisk_log_size":7624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5209,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7661","reported_seq":7784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.309759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:57.309759+0000","last_peered":"2024-09-15T21:20:57.309759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:57.309759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31363072,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7760","reported_seq":10071,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:57.312213+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:57.312213+0000","last_peered":"2024-09-15T21:20:57.312213+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:57.312213+0000","last_undegraded":"2024-09-15T21:20:57.312213+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7760,"log_dups_size":0,"ondisk_log_size":7760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31736425,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5420,"num_write_kb":21680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5370},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":501940841,"num_objects":122545,"num_object_clones":0,"num_object_copies":245090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":122545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94040,"num_write_kb":376160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4484,"num_bytes_recovered":18362368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1022812160,"data_stored":1022728664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122626,"ondisk_log_size":122626,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953516,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531568,"kb_used_data":351328,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93840272,"statfs":{"total":96636764160,"available":96092438528,"internally_reserved":0,"allocated":359759872,"data_stored":359494027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697172,"kb_used_data":422596,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674668,"statfs":{"total":96636764160,"available":95922860032,"internally_reserved":0,"allocated":432738304,"data_stored":432472459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626462,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422408,"kb_used_data":224120,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93949432,"statfs":{"total":96636764160,"available":96204218368,"internally_reserved":0,"allocated":229498880,"data_stored":229159976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":203026092},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433143808,"data_stored":433140329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":229687296,"data_stored":229610758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359981056,"data_stored":359977577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:00.530 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 187 20 124432 124412 2.59799 1.60156 0.0260493 0.0300646 2024-09-15T21:21:01.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 188 20 124956 124936 2.59503 2.04688 0.0211383 0.0300984 2024-09-15T21:21:02.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 189 20 125483 125463 2.59219 2.05859 0.0149599 0.03013 2024-09-15T21:21:03.345 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:03.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 190 20 125992 125972 2.58901 1.98828 0.0291794 0.0301689 2024-09-15T21:21:04.157 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:04.157 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:04.226 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-15T21:21:02.370755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505557628,"num_objects":123430,"num_object_clones":0,"num_object_copies":246860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":123430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94925,"num_write_kb":379694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4485,"num_bytes_recovered":18362387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123512,"ondisk_log_size":123512,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1689800,"kb_used_data":1010328,"kb_used_omap":12,"kb_used_meta":679411,"kb_avail":281425720,"statfs":{"total":289910292480,"available":288179937280,"internally_reserved":0,"allocated":1034575872,"data_stored":1033705278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":695716874},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,5,11,14,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20209664,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4934,"num_write_kb":19736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1241088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003022"},"pg_stats":[{"pgid":"2.f","version":"46'7632","reported_seq":7708,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.250417+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:20:57.250417+0000","last_peered":"2024-09-15T21:20:57.250417+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:20:57.250417+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7632,"log_dups_size":0,"ondisk_log_size":7632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31260672,"num_objects":7632,"num_object_clones":0,"num_object_copies":15264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":30528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4265},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7781","reported_seq":7877,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.010725+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:20:59.010725+0000","last_peered":"2024-09-15T21:20:59.010725+0000","last_clean":"2024-09-15T21:20:59.010725+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:20:59.010725+0000","last_undegraded":"2024-09-15T21:20:59.010725+0000","last_fullsized":"2024-09-15T21:20:59.010725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7781,"log_dups_size":0,"ondisk_log_size":7781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31846400,"num_objects":7775,"num_object_clones":0,"num_object_copies":15550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6507,"num_write_kb":26028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7915","reported_seq":14830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.268571+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:02.268571+0000","last_peered":"2024-09-15T21:21:02.268571+0000","last_clean":"2024-09-15T21:21:02.268571+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:02.268571+0000","last_undegraded":"2024-09-15T21:21:02.268571+0000","last_fullsized":"2024-09-15T21:21:02.268571+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7915,"log_dups_size":0,"ondisk_log_size":7915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6638,"num_write_kb":26552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7772","reported_seq":7841,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.083099+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:20:59.083099+0000","last_peered":"2024-09-15T21:20:59.083099+0000","last_clean":"2024-09-15T21:20:59.083099+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:20:59.083099+0000","last_undegraded":"2024-09-15T21:20:59.083099+0000","last_fullsized":"2024-09-15T21:20:59.083099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7772,"log_dups_size":0,"ondisk_log_size":7772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7772,"num_write_kb":31088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7817","reported_seq":7935,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.271887+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:20:57.271887+0000","last_peered":"2024-09-15T21:20:57.271887+0000","last_clean":"2024-09-15T21:20:57.271887+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:20:57.271887+0000","last_undegraded":"2024-09-15T21:20:57.271887+0000","last_fullsized":"2024-09-15T21:20:57.271887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7679","reported_seq":7808,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.304694+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:20:57.304694+0000","last_peered":"2024-09-15T21:20:57.304694+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:20:57.304694+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7679,"log_dups_size":0,"ondisk_log_size":7679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31432704,"num_objects":7674,"num_object_clones":0,"num_object_copies":15348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5104,"num_write_kb":20416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4303},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7903","reported_seq":8022,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.237010+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:20:57.237010+0000","last_peered":"2024-09-15T21:20:57.237010+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:20:57.237010+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32346112,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5539,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7609","reported_seq":7728,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.250469+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:20:57.250469+0000","last_peered":"2024-09-15T21:20:57.250469+0000","last_clean":"2024-09-15T21:20:57.250469+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:20:57.250469+0000","last_undegraded":"2024-09-15T21:20:57.250469+0000","last_fullsized":"2024-09-15T21:20:57.250469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7609,"log_dups_size":0,"ondisk_log_size":7609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31158272,"num_objects":7607,"num_object_clones":0,"num_object_copies":15214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5394,"num_write_kb":21576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7662","reported_seq":7783,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:58.992964+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:20:58.992964+0000","last_peered":"2024-09-15T21:20:58.992964+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:20:58.992964+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6248,"num_write_kb":24992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4317}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7541","reported_seq":7663,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.274840+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:20:57.274840+0000","last_peered":"2024-09-15T21:20:57.274840+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:20:57.274840+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30855168,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5067,"num_write_kb":20268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4192},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7606","reported_seq":7715,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:59.038123+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:20:59.038123+0000","last_peered":"2024-09-15T21:20:59.038123+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:20:59.038123+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7606,"log_dups_size":0,"ondisk_log_size":7606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4335}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7772","reported_seq":7886,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:59.096763+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:20:59.096763+0000","last_peered":"2024-09-15T21:20:59.096763+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:20:59.096763+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7772,"log_dups_size":0,"ondisk_log_size":7772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6171,"num_write_kb":24684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7775","reported_seq":7902,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.325116+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:20:57.325116+0000","last_peered":"2024-09-15T21:20:57.325116+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:20:57.325116+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7775,"log_dups_size":0,"ondisk_log_size":7775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31838208,"num_objects":7773,"num_object_clones":0,"num_object_copies":15546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7624","reported_seq":7753,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.282687+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:20:57.282687+0000","last_peered":"2024-09-15T21:20:57.282687+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:20:57.282687+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7624,"log_dups_size":0,"ondisk_log_size":7624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195136,"num_objects":7616,"num_object_clones":0,"num_object_copies":15232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5209,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4237},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7661","reported_seq":7784,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:20:57.309759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:20:57.309759+0000","last_peered":"2024-09-15T21:20:57.309759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:20:57.309759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31363072,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5308,"num_write_kb":21232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4292},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7760","reported_seq":10071,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:20:57.312213+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:20:57.312213+0000","last_peered":"2024-09-15T21:20:57.312213+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:20:57.312213+0000","last_undegraded":"2024-09-15T21:20:57.312213+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7760,"log_dups_size":0,"ondisk_log_size":7760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31736425,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5420,"num_write_kb":21680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5370},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505557609,"num_objects":123428,"num_object_clones":0,"num_object_copies":246856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2379,"num_objects_unfound":0,"num_objects_dirty":123428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94923,"num_write_kb":379692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4484,"num_bytes_recovered":18362368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1035689984,"data_stored":1035606488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123509,"ondisk_log_size":123509,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953517,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540500,"kb_used_data":356100,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93831340,"statfs":{"total":96636764160,"available":96083292160,"internally_reserved":0,"allocated":364646400,"data_stored":364380555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697172,"kb_used_data":422596,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674668,"statfs":{"total":96636764160,"available":95922860032,"internally_reserved":0,"allocated":432738304,"data_stored":432472459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626463,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452128,"kb_used_data":231632,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93919712,"statfs":{"total":96636764160,"available":96173785088,"internally_reserved":0,"allocated":237191168,"data_stored":236852264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,5,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":433143808,"data_stored":433140329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237551616,"data_stored":237475078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364994560,"data_stored":364991081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:04.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:04.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 191 20 126476 126456 2.58536 1.89062 0.0301777 0.0302083 2024-09-15T21:21:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:04.899 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:04.919 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-15T21:21:04.372437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514634364,"num_objects":125646,"num_object_clones":0,"num_object_copies":251292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":125646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97141,"num_write_kb":388558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4626,"num_bytes_recovered":18939923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125728,"ondisk_log_size":125728,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1685132,"kb_used_data":1022300,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281430388,"statfs":{"total":289910292480,"available":288184717312,"internally_reserved":0,"allocated":1046835200,"data_stored":1045964606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":678677514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,6,9,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22106112,"num_objects":5397,"num_object_clones":0,"num_object_copies":10794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":5397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004277"},"pg_stats":[{"pgid":"2.f","version":"46'7767","reported_seq":7843,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.326865+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:02.326865+0000","last_peered":"2024-09-15T21:21:02.326865+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:02.326865+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7767,"log_dups_size":0,"ondisk_log_size":7767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31813632,"num_objects":7767,"num_object_clones":0,"num_object_copies":15534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7767,"num_write_kb":31068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4400},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7931","reported_seq":8027,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:04.046968+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:04.046968+0000","last_peered":"2024-09-15T21:21:04.046968+0000","last_clean":"2024-09-15T21:21:04.046968+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:04.046968+0000","last_undegraded":"2024-09-15T21:21:04.046968+0000","last_fullsized":"2024-09-15T21:21:04.046968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7915","reported_seq":14830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.268571+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:02.268571+0000","last_peered":"2024-09-15T21:21:02.268571+0000","last_clean":"2024-09-15T21:21:02.268571+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:02.268571+0000","last_undegraded":"2024-09-15T21:21:02.268571+0000","last_fullsized":"2024-09-15T21:21:02.268571+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7915,"log_dups_size":0,"ondisk_log_size":7915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6638,"num_write_kb":26552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7919","reported_seq":7988,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:04.087068+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:04.087068+0000","last_peered":"2024-09-15T21:21:04.087068+0000","last_clean":"2024-09-15T21:21:04.087068+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:04.087068+0000","last_undegraded":"2024-09-15T21:21:04.087068+0000","last_fullsized":"2024-09-15T21:21:04.087068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7970","reported_seq":8088,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.248788+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:02.248788+0000","last_peered":"2024-09-15T21:21:02.248788+0000","last_clean":"2024-09-15T21:21:02.248788+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:02.248788+0000","last_undegraded":"2024-09-15T21:21:02.248788+0000","last_fullsized":"2024-09-15T21:21:02.248788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7970,"log_dups_size":0,"ondisk_log_size":7970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32616448,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7809","reported_seq":7937,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.304468+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:02.304468+0000","last_peered":"2024-09-15T21:21:02.304468+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:02.304468+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965184,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8053","reported_seq":8176,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.283172+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:02.283172+0000","last_peered":"2024-09-15T21:21:02.283172+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:02.283172+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7761","reported_seq":7880,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.296074+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:02.296074+0000","last_peered":"2024-09-15T21:21:02.296074+0000","last_clean":"2024-09-15T21:21:02.296074+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:02.296074+0000","last_undegraded":"2024-09-15T21:21:02.296074+0000","last_fullsized":"2024-09-15T21:21:02.296074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7799","reported_seq":7923,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:04.099624+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:04.099624+0000","last_peered":"2024-09-15T21:21:04.099624+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:04.099624+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7799,"log_dups_size":0,"ondisk_log_size":7799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6385,"num_write_kb":25540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4454}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7674","reported_seq":7796,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.119138+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:02.119138+0000","last_peered":"2024-09-15T21:21:02.119138+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:02.119138+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31399936,"num_objects":7666,"num_object_clones":0,"num_object_copies":15332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4325},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7737","reported_seq":7846,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:03.988129+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:03.988129+0000","last_peered":"2024-09-15T21:21:03.988129+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:03.988129+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7737,"log_dups_size":0,"ondisk_log_size":7737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4466}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7942","reported_seq":8056,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:04.007514+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:04.007514+0000","last_peered":"2024-09-15T21:21:04.007514+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:04.007514+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7942,"log_dups_size":0,"ondisk_log_size":7942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32505856,"num_objects":7936,"num_object_clones":0,"num_object_copies":15872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6341,"num_write_kb":25364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7931","reported_seq":8059,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.358150+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:02.358150+0000","last_peered":"2024-09-15T21:21:02.358150+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:02.358150+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5238,"num_write_kb":20952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4496},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7800","reported_seq":7929,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.339695+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:02.339695+0000","last_peered":"2024-09-15T21:21:02.339695+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:02.339695+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7800,"log_dups_size":0,"ondisk_log_size":7800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5385,"num_write_kb":21540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4413},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7809","reported_seq":7932,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.312991+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:02.312991+0000","last_peered":"2024-09-15T21:21:02.312991+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:02.312991+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969280,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4440},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7908","reported_seq":10500,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:02.261591+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:02.261591+0000","last_peered":"2024-09-15T21:21:02.261591+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:02.261591+0000","last_undegraded":"2024-09-15T21:21:02.261591+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7908,"log_dups_size":0,"ondisk_log_size":7908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32342633,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5568,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1232,"num_bytes_recovered":5046272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5659},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514634345,"num_objects":125644,"num_object_clones":0,"num_object_copies":251288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":125644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97139,"num_write_kb":388556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4625,"num_bytes_recovered":18939904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1047883776,"data_stored":1047800280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125725,"ondisk_log_size":125725,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537092,"kb_used_data":361012,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93834748,"statfs":{"total":96636764160,"available":96086781952,"internally_reserved":0,"allocated":369676288,"data_stored":369410443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695912,"kb_used_data":429656,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93675928,"statfs":{"total":96636764160,"available":95924150272,"internally_reserved":0,"allocated":439967744,"data_stored":439701899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":272625329},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,4,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626463,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452128,"kb_used_data":231632,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93919712,"statfs":{"total":96636764160,"available":96173785088,"internally_reserved":0,"allocated":237191168,"data_stored":236852264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,5,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237551616,"data_stored":237475078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369909760,"data_stored":369906281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7767", 'reported_seq': 7843, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.326865+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:02.326865+0000', 'last_peered': '2024-09-15T21:21:02.326865+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:02.326865+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7767, 'log_dups_size': 0, 'ondisk_log_size': 7767, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31813632, 'num_objects': 7767, 'num_object_clones': 0, 'num_object_copies': 15534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7767, 'num_write_kb': 31068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4400}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7809", 'reported_seq': 7937, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.304468+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:02.304468+0000', 'last_peered': '2024-09-15T21:21:02.304468+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:02.304468+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7809, 'log_dups_size': 0, 'ondisk_log_size': 7809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31965184, 'num_objects': 7804, 'num_object_clones': 0, 'num_object_copies': 15608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5234, 'num_write_kb': 20936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4433}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8053", 'reported_seq': 8176, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.283172+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:02.283172+0000', 'last_peered': '2024-09-15T21:21:02.283172+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:02.283172+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8053, 'log_dups_size': 0, 'ondisk_log_size': 8053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32960512, 'num_objects': 8047, 'num_object_clones': 0, 'num_object_copies': 16094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5689, 'num_write_kb': 22756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4576}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:04.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'7799", 'reported_seq': 7923, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:04.099624+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:04.099624+0000', 'last_peered': '2024-09-15T21:21:04.099624+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:04.099624+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 7799, 'log_dups_size': 0, 'ondisk_log_size': 7799, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31916032, 'num_objects': 7792, 'num_object_clones': 0, 'num_object_copies': 15584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6385, 'num_write_kb': 25540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'7674", 'reported_seq': 7796, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.119138+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:02.119138+0000', 'last_peered': '2024-09-15T21:21:02.119138+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:02.119138+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 7674, 'log_dups_size': 0, 'ondisk_log_size': 7674, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31399936, 'num_objects': 7666, 'num_object_clones': 0, 'num_object_copies': 15332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5200, 'num_write_kb': 20800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4325}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'7737", 'reported_seq': 7846, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:03.988129+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:03.988129+0000', 'last_peered': '2024-09-15T21:21:03.988129+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:03.988129+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 7737, 'log_dups_size': 0, 'ondisk_log_size': 7737, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31670272, 'num_objects': 7732, 'num_object_clones': 0, 'num_object_copies': 15464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6288, 'num_write_kb': 25152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 4466}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'7942", 'reported_seq': 8056, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:04.007514+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:04.007514+0000', 'last_peered': '2024-09-15T21:21:04.007514+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:04.007514+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 7942, 'log_dups_size': 0, 'ondisk_log_size': 7942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32505856, 'num_objects': 7936, 'num_object_clones': 0, 'num_object_copies': 15872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6341, 'num_write_kb': 25364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4586}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'7931", 'reported_seq': 8059, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.358150+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:02.358150+0000', 'last_peered': '2024-09-15T21:21:02.358150+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:02.358150+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 7931, 'log_dups_size': 0, 'ondisk_log_size': 7931, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32477184, 'num_objects': 7929, 'num_object_clones': 0, 'num_object_copies': 15858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5238, 'num_write_kb': 20952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4496}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:04.922 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'7800", 'reported_seq': 7929, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.339695+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:02.339695+0000', 'last_peered': '2024-09-15T21:21:02.339695+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:02.339695+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 7800, 'log_dups_size': 0, 'ondisk_log_size': 7800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31916032, 'num_objects': 7792, 'num_object_clones': 0, 'num_object_copies': 15584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5385, 'num_write_kb': 21540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4413}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:04.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7809", 'reported_seq': 7932, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:02.312991+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:02.312991+0000', 'last_peered': '2024-09-15T21:21:02.312991+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:02.312991+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7809, 'log_dups_size': 0, 'ondisk_log_size': 7809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31969280, 'num_objects': 7805, 'num_object_clones': 0, 'num_object_copies': 15610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5456, 'num_write_kb': 21824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4440}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:04.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'7908", 'reported_seq': 10500, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:02.261591+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:02.261591+0000', 'last_peered': '2024-09-15T21:21:02.261591+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:02.261591+0000', 'last_undegraded': '2024-09-15T21:21:02.261591+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 7908, 'log_dups_size': 0, 'ondisk_log_size': 7908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32342633, 'num_objects': 7897, 'num_object_clones': 0, 'num_object_copies': 15794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2238, 'num_objects_unfound': 0, 'num_objects_dirty': 7897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5568, 'num_write_kb': 22272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1232, 'num_bytes_recovered': 5046272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5659}, {'shards': '0,2', 'objects': 2238}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:04.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:05.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 192 20 126861 126841 2.57972 1.50391 0.0924693 0.0302772 2024-09-15T21:21:05.828 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:05.829 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/251292 objects degraded (13.410%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125646,"data_bytes":514634364,"bytes_used":1725575168,"bytes_avail":288184717312,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":251292,"degraded_ratio":0.13410295592378588,"misplaced_objects":2238,"misplaced_total":251292,"misplaced_ratio":0.0089059739267465729,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109528,"recovering_keys_per_sec":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"write_bytes_sec":1841519,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:05.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:05.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:21:06.337+0000 7f56feffd640 0 --2- 172.21.15.114:0/2966625037 >> [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] conn(0x7f56f0004070 0x7f56f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:21:06.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 193 20 127348 127328 2.57621 1.90234 0.0664754 0.0303161 2024-09-15T21:21:07.354 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:07.355 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:07.420 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-15T21:21:06.372938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514634364,"num_objects":125646,"num_object_clones":0,"num_object_copies":251292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":125646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97141,"num_write_kb":388558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4626,"num_bytes_recovered":18939923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125728,"ondisk_log_size":125728,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1685132,"kb_used_data":1022300,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281430388,"statfs":{"total":289910292480,"available":288184717312,"internally_reserved":0,"allocated":1046835200,"data_stored":1045964606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":678677514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,6,9,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19087360,"num_objects":4660,"num_object_clones":0,"num_object_copies":9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":18640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004331"},"pg_stats":[{"pgid":"2.f","version":"46'7767","reported_seq":7843,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.326865+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:02.326865+0000","last_peered":"2024-09-15T21:21:02.326865+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:02.326865+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7767,"log_dups_size":0,"ondisk_log_size":7767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31813632,"num_objects":7767,"num_object_clones":0,"num_object_copies":15534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7767,"num_write_kb":31068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4400},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'7931","reported_seq":8027,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:04.046968+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:04.046968+0000","last_peered":"2024-09-15T21:21:04.046968+0000","last_clean":"2024-09-15T21:21:04.046968+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:04.046968+0000","last_undegraded":"2024-09-15T21:21:04.046968+0000","last_fullsized":"2024-09-15T21:21:04.046968+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'7915","reported_seq":14830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.268571+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:02.268571+0000","last_peered":"2024-09-15T21:21:02.268571+0000","last_clean":"2024-09-15T21:21:02.268571+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:02.268571+0000","last_undegraded":"2024-09-15T21:21:02.268571+0000","last_fullsized":"2024-09-15T21:21:02.268571+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":7915,"log_dups_size":0,"ondisk_log_size":7915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6638,"num_write_kb":26552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'7919","reported_seq":7988,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:04.087068+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:04.087068+0000","last_peered":"2024-09-15T21:21:04.087068+0000","last_clean":"2024-09-15T21:21:04.087068+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:04.087068+0000","last_undegraded":"2024-09-15T21:21:04.087068+0000","last_fullsized":"2024-09-15T21:21:04.087068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7919,"log_dups_size":0,"ondisk_log_size":7919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7970","reported_seq":8088,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.248788+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:02.248788+0000","last_peered":"2024-09-15T21:21:02.248788+0000","last_clean":"2024-09-15T21:21:02.248788+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:02.248788+0000","last_undegraded":"2024-09-15T21:21:02.248788+0000","last_fullsized":"2024-09-15T21:21:02.248788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":7970,"log_dups_size":0,"ondisk_log_size":7970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32616448,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7809","reported_seq":7937,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.304468+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:02.304468+0000","last_peered":"2024-09-15T21:21:02.304468+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:02.304468+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965184,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5234,"num_write_kb":20936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4433},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8053","reported_seq":8176,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.283172+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:02.283172+0000","last_peered":"2024-09-15T21:21:02.283172+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:02.283172+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5689,"num_write_kb":22756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4576},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7761","reported_seq":7880,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:02.296074+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:02.296074+0000","last_peered":"2024-09-15T21:21:02.296074+0000","last_clean":"2024-09-15T21:21:02.296074+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:02.296074+0000","last_undegraded":"2024-09-15T21:21:02.296074+0000","last_fullsized":"2024-09-15T21:21:02.296074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7761,"log_dups_size":0,"ondisk_log_size":7761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5546,"num_write_kb":22184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7799","reported_seq":7923,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:04.099624+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:04.099624+0000","last_peered":"2024-09-15T21:21:04.099624+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:04.099624+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7799,"log_dups_size":0,"ondisk_log_size":7799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6385,"num_write_kb":25540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4454}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7674","reported_seq":7796,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.119138+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:02.119138+0000","last_peered":"2024-09-15T21:21:02.119138+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:02.119138+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31399936,"num_objects":7666,"num_object_clones":0,"num_object_copies":15332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5200,"num_write_kb":20800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4325},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7737","reported_seq":7846,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:03.988129+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:03.988129+0000","last_peered":"2024-09-15T21:21:03.988129+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:03.988129+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7737,"log_dups_size":0,"ondisk_log_size":7737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4466}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'7942","reported_seq":8056,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:04.007514+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:04.007514+0000","last_peered":"2024-09-15T21:21:04.007514+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:04.007514+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":7942,"log_dups_size":0,"ondisk_log_size":7942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32505856,"num_objects":7936,"num_object_clones":0,"num_object_copies":15872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6341,"num_write_kb":25364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'7931","reported_seq":8059,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.358150+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:02.358150+0000","last_peered":"2024-09-15T21:21:02.358150+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:02.358150+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32477184,"num_objects":7929,"num_object_clones":0,"num_object_copies":15858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5238,"num_write_kb":20952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4496},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7800","reported_seq":7929,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.339695+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:02.339695+0000","last_peered":"2024-09-15T21:21:02.339695+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:02.339695+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7800,"log_dups_size":0,"ondisk_log_size":7800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5385,"num_write_kb":21540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4413},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7809","reported_seq":7932,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:02.312991+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:02.312991+0000","last_peered":"2024-09-15T21:21:02.312991+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:02.312991+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969280,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4440},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7908","reported_seq":10500,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:02.261591+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:02.261591+0000","last_peered":"2024-09-15T21:21:02.261591+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:02.261591+0000","last_undegraded":"2024-09-15T21:21:02.261591+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":7908,"log_dups_size":0,"ondisk_log_size":7908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32342633,"num_objects":7897,"num_object_clones":0,"num_object_copies":15794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":7897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5568,"num_write_kb":22272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1232,"num_bytes_recovered":5046272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5659},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514634345,"num_objects":125644,"num_object_clones":0,"num_object_copies":251288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2238,"num_objects_unfound":0,"num_objects_dirty":125644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97139,"num_write_kb":388556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4625,"num_bytes_recovered":18939904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1047883776,"data_stored":1047800280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125725,"ondisk_log_size":125725,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953518,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537092,"kb_used_data":361012,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93834748,"statfs":{"total":96636764160,"available":96086781952,"internally_reserved":0,"allocated":369676288,"data_stored":369410443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149745,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695912,"kb_used_data":429656,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93675928,"statfs":{"total":96636764160,"available":95924150272,"internally_reserved":0,"allocated":439967744,"data_stored":439701899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":272625329},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,4,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626463,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452128,"kb_used_data":231632,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93919712,"statfs":{"total":96636764160,"available":96173785088,"internally_reserved":0,"allocated":237191168,"data_stored":236852264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,5,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237551616,"data_stored":237475078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369909760,"data_stored":369906281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:07.532 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 194 20 127854 127834 2.57312 1.97656 0.0288137 0.0303536 2024-09-15T21:21:08.533 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 195 20 128389 128369 2.57064 2.08984 0.0490405 0.0303835 2024-09-15T21:21:09.533 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 196 20 128800 128780 2.56572 1.60547 0.0208697 0.0304396 2024-09-15T21:21:10.423 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:10.533 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 197 20 129213 129193 2.56089 1.61328 0.0346687 0.0304984 2024-09-15T21:21:11.461 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:11.464 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:11.525 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-15T21:21:10.373873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524608124,"num_objects":128081,"num_object_clones":0,"num_object_copies":256162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99576,"num_write_kb":398298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4771,"num_bytes_recovered":19533843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128163,"ondisk_log_size":128163,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704884,"kb_used_data":1042052,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281410636,"statfs":{"total":289910292480,"available":288164491264,"internally_reserved":0,"allocated":1067061248,"data_stored":1066190654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":678677514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,4,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22667264,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5534,"num_write_kb":22136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004274"},"pg_stats":[{"pgid":"2.f","version":"46'7927","reported_seq":8003,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.401463+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:07.401463+0000","last_peered":"2024-09-15T21:21:07.401463+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:07.401463+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":31708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8086","reported_seq":8182,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.080215+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:09.080215+0000","last_peered":"2024-09-15T21:21:09.080215+0000","last_clean":"2024-09-15T21:21:09.080215+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:09.080215+0000","last_undegraded":"2024-09-15T21:21:09.080215+0000","last_fullsized":"2024-09-15T21:21:09.080215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8077","reported_seq":14992,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.335810+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:07.335810+0000","last_peered":"2024-09-15T21:21:07.335810+0000","last_clean":"2024-09-15T21:21:07.335810+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:07.335810+0000","last_undegraded":"2024-09-15T21:21:07.335810+0000","last_fullsized":"2024-09-15T21:21:07.335810+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067008,"num_objects":8073,"num_object_clones":0,"num_object_copies":16146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6800,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8068","reported_seq":8135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.017182+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:09.017182+0000","last_peered":"2024-09-15T21:21:09.017182+0000","last_clean":"2024-09-15T21:21:09.017182+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:09.017182+0000","last_undegraded":"2024-09-15T21:21:09.017182+0000","last_fullsized":"2024-09-15T21:21:09.017182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8068,"log_dups_size":0,"ondisk_log_size":8068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8122","reported_seq":8240,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.373368+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:07.373368+0000","last_peered":"2024-09-15T21:21:07.373368+0000","last_clean":"2024-09-15T21:21:07.373368+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:07.373368+0000","last_undegraded":"2024-09-15T21:21:07.373368+0000","last_fullsized":"2024-09-15T21:21:07.373368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8122,"log_dups_size":0,"ondisk_log_size":8122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7963","reported_seq":8092,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426420+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:07.426420+0000","last_peered":"2024-09-15T21:21:07.426420+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:07.426420+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7963,"log_dups_size":0,"ondisk_log_size":7963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8211","reported_seq":8334,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.433494+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:07.433494+0000","last_peered":"2024-09-15T21:21:07.433494+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:07.433494+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4734},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7929","reported_seq":8048,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.361829+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:07.361829+0000","last_peered":"2024-09-15T21:21:07.361829+0000","last_clean":"2024-09-15T21:21:07.361829+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:07.361829+0000","last_undegraded":"2024-09-15T21:21:07.361829+0000","last_fullsized":"2024-09-15T21:21:07.361829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7929,"log_dups_size":0,"ondisk_log_size":7929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5714,"num_write_kb":22856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7948","reported_seq":8072,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.033980+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:09.033980+0000","last_peered":"2024-09-15T21:21:09.033980+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:09.033980+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7948,"log_dups_size":0,"ondisk_log_size":7948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526336,"num_objects":7941,"num_object_clones":0,"num_object_copies":15882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7831","reported_seq":7952,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.416763+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:07.416763+0000","last_peered":"2024-09-15T21:21:07.416763+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:07.416763+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7831,"log_dups_size":0,"ondisk_log_size":7831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7892","reported_seq":8001,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.108497+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:09.108497+0000","last_peered":"2024-09-15T21:21:09.108497+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:09.108497+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7892,"log_dups_size":0,"ondisk_log_size":7892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8073","reported_seq":8187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.064286+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:09.064286+0000","last_peered":"2024-09-15T21:21:09.064286+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:09.064286+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33042432,"num_objects":8067,"num_object_clones":0,"num_object_copies":16134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8085","reported_seq":8213,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.424620+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:07.424620+0000","last_peered":"2024-09-15T21:21:07.424620+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:07.424620+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33107968,"num_objects":8083,"num_object_clones":0,"num_object_copies":16166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":21568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4650},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7947","reported_seq":8075,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426862+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:07.426862+0000","last_peered":"2024-09-15T21:21:07.426862+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:07.426862+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32518144,"num_objects":7939,"num_object_clones":0,"num_object_copies":15878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7950","reported_seq":8073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.423295+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:07.423295+0000","last_peered":"2024-09-15T21:21:07.423295+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:07.423295+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8051","reported_seq":10934,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:07.426034+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:07.426034+0000","last_peered":"2024-09-15T21:21:07.426034+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:07.426034+0000","last_undegraded":"2024-09-15T21:21:07.426034+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928361,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5947},{"shards":"0,2","objects":2093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524608105,"num_objects":128079,"num_object_clones":0,"num_object_copies":256158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99574,"num_write_kb":398296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4770,"num_bytes_recovered":19533824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068457984,"data_stored":1068374488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128160,"ondisk_log_size":128160,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953519,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545996,"kb_used_data":365756,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93825844,"statfs":{"total":96636764160,"available":96077664256,"internally_reserved":0,"allocated":374534144,"data_stored":374268299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707292,"kb_used_data":436876,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93664548,"statfs":{"total":96636764160,"available":95912497152,"internally_reserved":0,"allocated":447361024,"data_stored":447095179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,2,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451596,"kb_used_data":239420,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93920244,"statfs":{"total":96636764160,"available":96174329856,"internally_reserved":0,"allocated":245166080,"data_stored":244827176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447877120,"data_stored":447873641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245596160,"data_stored":245519622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374984704,"data_stored":374981225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:11.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:11.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 198 20 129702 129682 2.55756 1.91016 0.013823 0.0305382 2024-09-15T21:21:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:12.387 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:12.447 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-15T21:21:12.377490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524608124,"num_objects":128081,"num_object_clones":0,"num_object_copies":256162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99576,"num_write_kb":398298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4771,"num_bytes_recovered":19533843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128163,"ondisk_log_size":128163,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704884,"kb_used_data":1042052,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281410636,"statfs":{"total":289910292480,"available":288164491264,"internally_reserved":0,"allocated":1067061248,"data_stored":1066190654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":678677514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,4,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4813,"num_write_kb":19252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007399"},"pg_stats":[{"pgid":"2.f","version":"46'7927","reported_seq":8003,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.401463+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:07.401463+0000","last_peered":"2024-09-15T21:21:07.401463+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:07.401463+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":31708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8086","reported_seq":8182,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.080215+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:09.080215+0000","last_peered":"2024-09-15T21:21:09.080215+0000","last_clean":"2024-09-15T21:21:09.080215+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:09.080215+0000","last_undegraded":"2024-09-15T21:21:09.080215+0000","last_fullsized":"2024-09-15T21:21:09.080215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8077","reported_seq":14992,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.335810+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:07.335810+0000","last_peered":"2024-09-15T21:21:07.335810+0000","last_clean":"2024-09-15T21:21:07.335810+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:07.335810+0000","last_undegraded":"2024-09-15T21:21:07.335810+0000","last_fullsized":"2024-09-15T21:21:07.335810+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067008,"num_objects":8073,"num_object_clones":0,"num_object_copies":16146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6800,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8068","reported_seq":8135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.017182+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:09.017182+0000","last_peered":"2024-09-15T21:21:09.017182+0000","last_clean":"2024-09-15T21:21:09.017182+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:09.017182+0000","last_undegraded":"2024-09-15T21:21:09.017182+0000","last_fullsized":"2024-09-15T21:21:09.017182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8068,"log_dups_size":0,"ondisk_log_size":8068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8122","reported_seq":8240,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.373368+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:07.373368+0000","last_peered":"2024-09-15T21:21:07.373368+0000","last_clean":"2024-09-15T21:21:07.373368+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:07.373368+0000","last_undegraded":"2024-09-15T21:21:07.373368+0000","last_fullsized":"2024-09-15T21:21:07.373368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8122,"log_dups_size":0,"ondisk_log_size":8122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7963","reported_seq":8092,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426420+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:07.426420+0000","last_peered":"2024-09-15T21:21:07.426420+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:07.426420+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7963,"log_dups_size":0,"ondisk_log_size":7963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8211","reported_seq":8334,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.433494+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:07.433494+0000","last_peered":"2024-09-15T21:21:07.433494+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:07.433494+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4734},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7929","reported_seq":8048,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.361829+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:07.361829+0000","last_peered":"2024-09-15T21:21:07.361829+0000","last_clean":"2024-09-15T21:21:07.361829+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:07.361829+0000","last_undegraded":"2024-09-15T21:21:07.361829+0000","last_fullsized":"2024-09-15T21:21:07.361829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7929,"log_dups_size":0,"ondisk_log_size":7929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5714,"num_write_kb":22856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7948","reported_seq":8072,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.033980+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:09.033980+0000","last_peered":"2024-09-15T21:21:09.033980+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:09.033980+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7948,"log_dups_size":0,"ondisk_log_size":7948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526336,"num_objects":7941,"num_object_clones":0,"num_object_copies":15882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7831","reported_seq":7952,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.416763+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:07.416763+0000","last_peered":"2024-09-15T21:21:07.416763+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:07.416763+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7831,"log_dups_size":0,"ondisk_log_size":7831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7892","reported_seq":8001,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.108497+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:09.108497+0000","last_peered":"2024-09-15T21:21:09.108497+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:09.108497+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7892,"log_dups_size":0,"ondisk_log_size":7892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8073","reported_seq":8187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.064286+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:09.064286+0000","last_peered":"2024-09-15T21:21:09.064286+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:09.064286+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33042432,"num_objects":8067,"num_object_clones":0,"num_object_copies":16134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8085","reported_seq":8213,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.424620+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:07.424620+0000","last_peered":"2024-09-15T21:21:07.424620+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:07.424620+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33107968,"num_objects":8083,"num_object_clones":0,"num_object_copies":16166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":21568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4650},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7947","reported_seq":8075,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426862+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:07.426862+0000","last_peered":"2024-09-15T21:21:07.426862+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:07.426862+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32518144,"num_objects":7939,"num_object_clones":0,"num_object_copies":15878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7950","reported_seq":8073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.423295+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:07.423295+0000","last_peered":"2024-09-15T21:21:07.423295+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:07.423295+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8051","reported_seq":10934,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:07.426034+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:07.426034+0000","last_peered":"2024-09-15T21:21:07.426034+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:07.426034+0000","last_undegraded":"2024-09-15T21:21:07.426034+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928361,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5947},{"shards":"0,2","objects":2093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524608105,"num_objects":128079,"num_object_clones":0,"num_object_copies":256158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99574,"num_write_kb":398296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4770,"num_bytes_recovered":19533824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068457984,"data_stored":1068374488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128160,"ondisk_log_size":128160,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953519,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545996,"kb_used_data":365756,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93825844,"statfs":{"total":96636764160,"available":96077664256,"internally_reserved":0,"allocated":374534144,"data_stored":374268299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707292,"kb_used_data":436876,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93664548,"statfs":{"total":96636764160,"available":95912497152,"internally_reserved":0,"allocated":447361024,"data_stored":447095179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,2,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451596,"kb_used_data":239420,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93920244,"statfs":{"total":96636764160,"available":96174329856,"internally_reserved":0,"allocated":245166080,"data_stored":244827176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447877120,"data_stored":447873641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245596160,"data_stored":245519622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374984704,"data_stored":374981225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'7927", 'reported_seq': 8003, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.401463+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:07.401463+0000', 'last_peered': '2024-09-15T21:21:07.401463+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:07.401463+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 7927, 'log_dups_size': 0, 'ondisk_log_size': 7927, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32468992, 'num_objects': 7927, 'num_object_clones': 0, 'num_object_copies': 15854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7927, 'num_write_kb': 31708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4560}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'7963", 'reported_seq': 8092, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.426420+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:07.426420+0000', 'last_peered': '2024-09-15T21:21:07.426420+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:07.426420+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 7963, 'log_dups_size': 0, 'ondisk_log_size': 7963, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32595968, 'num_objects': 7958, 'num_object_clones': 0, 'num_object_copies': 15916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5388, 'num_write_kb': 21552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4587}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8211", 'reported_seq': 8334, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.433494+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:07.433494+0000', 'last_peered': '2024-09-15T21:21:07.433494+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:07.433494+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8211, 'log_dups_size': 0, 'ondisk_log_size': 8211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33607680, 'num_objects': 8205, 'num_object_clones': 0, 'num_object_copies': 16410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5847, 'num_write_kb': 23388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4734}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'7948", 'reported_seq': 8072, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:09.033980+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:09.033980+0000', 'last_peered': '2024-09-15T21:21:09.033980+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:09.033980+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 7948, 'log_dups_size': 0, 'ondisk_log_size': 7948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32526336, 'num_objects': 7941, 'num_object_clones': 0, 'num_object_copies': 15882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6534, 'num_write_kb': 26136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 4603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:12.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'7831", 'reported_seq': 7952, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.416763+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:07.416763+0000', 'last_peered': '2024-09-15T21:21:07.416763+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:07.416763+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 7831, 'log_dups_size': 0, 'ondisk_log_size': 7831, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32043008, 'num_objects': 7823, 'num_object_clones': 0, 'num_object_copies': 15646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5357, 'num_write_kb': 21428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4482}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'7892", 'reported_seq': 8001, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:09.108497+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:09.108497+0000', 'last_peered': '2024-09-15T21:21:09.108497+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:09.108497+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 7892, 'log_dups_size': 0, 'ondisk_log_size': 7892, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32305152, 'num_objects': 7887, 'num_object_clones': 0, 'num_object_copies': 15774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6443, 'num_write_kb': 25772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 4621}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8073", 'reported_seq': 8187, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:09.064286+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:09.064286+0000', 'last_peered': '2024-09-15T21:21:09.064286+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:09.064286+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8073, 'log_dups_size': 0, 'ondisk_log_size': 8073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33042432, 'num_objects': 8067, 'num_object_clones': 0, 'num_object_copies': 16134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6472, 'num_write_kb': 25888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4717}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8085", 'reported_seq': 8213, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.424620+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:07.424620+0000', 'last_peered': '2024-09-15T21:21:07.424620+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:07.424620+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8085, 'log_dups_size': 0, 'ondisk_log_size': 8085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33107968, 'num_objects': 8083, 'num_object_clones': 0, 'num_object_copies': 16166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8083, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5392, 'num_write_kb': 21568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4650}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'7947", 'reported_seq': 8075, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.426862+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:07.426862+0000', 'last_peered': '2024-09-15T21:21:07.426862+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:07.426862+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 7947, 'log_dups_size': 0, 'ondisk_log_size': 7947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32518144, 'num_objects': 7939, 'num_object_clones': 0, 'num_object_copies': 15878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7939, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5532, 'num_write_kb': 22128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4560}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'7950", 'reported_seq': 8073, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:07.423295+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:07.423295+0000', 'last_peered': '2024-09-15T21:21:07.423295+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:07.423295+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 7950, 'log_dups_size': 0, 'ondisk_log_size': 7950, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32546816, 'num_objects': 7946, 'num_object_clones': 0, 'num_object_copies': 15892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5597, 'num_write_kb': 22388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4581}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:12.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8051", 'reported_seq': 10934, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:07.426034+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:07.426034+0000', 'last_peered': '2024-09-15T21:21:07.426034+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:07.426034+0000', 'last_undegraded': '2024-09-15T21:21:07.426034+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8051, 'log_dups_size': 0, 'ondisk_log_size': 8051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32928361, 'num_objects': 8040, 'num_object_clones': 0, 'num_object_copies': 16080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2093, 'num_objects_unfound': 0, 'num_objects_dirty': 8040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5711, 'num_write_kb': 22844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1377, 'num_bytes_recovered': 5640192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5947}, {'shards': '0,2', 'objects': 2093}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:12.451 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:12.541 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 199 20 130175 130155 2.55399 1.84766 0.0383361 0.0305817 2024-09-15T21:21:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:13.388 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/256162 objects degraded (13.155%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":286,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128081,"data_bytes":524608124,"bytes_used":1745801216,"bytes_avail":288164491264,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":256162,"degraded_ratio":0.13155347006972151,"misplaced_objects":2093,"misplaced_total":256162,"misplaced_ratio":0.008170610785362388,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97561,"recovering_keys_per_sec":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"write_bytes_sec":1641824,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:13.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:13.426 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:13.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:21:13.535943+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0306215 2024-09-15T21:21:13.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:21:13.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 200 20 130643 130623 2.55036 1.82812 0.0158753 0.0306215 2024-09-15T21:21:14.178 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:14.179 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:14.298 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-15T21:21:12.377490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524608124,"num_objects":128081,"num_object_clones":0,"num_object_copies":256162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99576,"num_write_kb":398298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4771,"num_bytes_recovered":19533843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128163,"ondisk_log_size":128163,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704884,"kb_used_data":1042052,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281410636,"statfs":{"total":289910292480,"available":288164491264,"internally_reserved":0,"allocated":1067061248,"data_stored":1066190654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":678677514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,4,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4813,"num_write_kb":19252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007399"},"pg_stats":[{"pgid":"2.f","version":"46'7927","reported_seq":8003,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.401463+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:07.401463+0000","last_peered":"2024-09-15T21:21:07.401463+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:07.401463+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":7927,"log_dups_size":0,"ondisk_log_size":7927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":31708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8086","reported_seq":8182,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.080215+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:09.080215+0000","last_peered":"2024-09-15T21:21:09.080215+0000","last_clean":"2024-09-15T21:21:09.080215+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:09.080215+0000","last_undegraded":"2024-09-15T21:21:09.080215+0000","last_fullsized":"2024-09-15T21:21:09.080215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8086,"log_dups_size":0,"ondisk_log_size":8086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6812,"num_write_kb":27248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8077","reported_seq":14992,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.335810+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:07.335810+0000","last_peered":"2024-09-15T21:21:07.335810+0000","last_clean":"2024-09-15T21:21:07.335810+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:07.335810+0000","last_undegraded":"2024-09-15T21:21:07.335810+0000","last_fullsized":"2024-09-15T21:21:07.335810+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067008,"num_objects":8073,"num_object_clones":0,"num_object_copies":16146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6800,"num_write_kb":27200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8068","reported_seq":8135,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:09.017182+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:09.017182+0000","last_peered":"2024-09-15T21:21:09.017182+0000","last_clean":"2024-09-15T21:21:09.017182+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:09.017182+0000","last_undegraded":"2024-09-15T21:21:09.017182+0000","last_fullsized":"2024-09-15T21:21:09.017182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8068,"log_dups_size":0,"ondisk_log_size":8068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8122","reported_seq":8240,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.373368+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:07.373368+0000","last_peered":"2024-09-15T21:21:07.373368+0000","last_clean":"2024-09-15T21:21:07.373368+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:07.373368+0000","last_undegraded":"2024-09-15T21:21:07.373368+0000","last_fullsized":"2024-09-15T21:21:07.373368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8122,"log_dups_size":0,"ondisk_log_size":8122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5980,"num_write_kb":23920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7963","reported_seq":8092,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426420+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:07.426420+0000","last_peered":"2024-09-15T21:21:07.426420+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:07.426420+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":7963,"log_dups_size":0,"ondisk_log_size":7963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8211","reported_seq":8334,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.433494+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:07.433494+0000","last_peered":"2024-09-15T21:21:07.433494+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:07.433494+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4734},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7929","reported_seq":8048,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:07.361829+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:07.361829+0000","last_peered":"2024-09-15T21:21:07.361829+0000","last_clean":"2024-09-15T21:21:07.361829+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:07.361829+0000","last_undegraded":"2024-09-15T21:21:07.361829+0000","last_fullsized":"2024-09-15T21:21:07.361829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":7929,"log_dups_size":0,"ondisk_log_size":7929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32468992,"num_objects":7927,"num_object_clones":0,"num_object_copies":15854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5714,"num_write_kb":22856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'7948","reported_seq":8072,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.033980+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:09.033980+0000","last_peered":"2024-09-15T21:21:09.033980+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:09.033980+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":7948,"log_dups_size":0,"ondisk_log_size":7948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526336,"num_objects":7941,"num_object_clones":0,"num_object_copies":15882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7831","reported_seq":7952,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.416763+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:07.416763+0000","last_peered":"2024-09-15T21:21:07.416763+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:07.416763+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7831,"log_dups_size":0,"ondisk_log_size":7831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'7892","reported_seq":8001,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.108497+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:09.108497+0000","last_peered":"2024-09-15T21:21:09.108497+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:09.108497+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":7892,"log_dups_size":0,"ondisk_log_size":7892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6443,"num_write_kb":25772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4621}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8073","reported_seq":8187,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:09.064286+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:09.064286+0000","last_peered":"2024-09-15T21:21:09.064286+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:09.064286+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8073,"log_dups_size":0,"ondisk_log_size":8073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33042432,"num_objects":8067,"num_object_clones":0,"num_object_copies":16134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4717}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8085","reported_seq":8213,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.424620+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:07.424620+0000","last_peered":"2024-09-15T21:21:07.424620+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:07.424620+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33107968,"num_objects":8083,"num_object_clones":0,"num_object_copies":16166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":21568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4650},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'7947","reported_seq":8075,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.426862+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:07.426862+0000","last_peered":"2024-09-15T21:21:07.426862+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:07.426862+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32518144,"num_objects":7939,"num_object_clones":0,"num_object_copies":15878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4560},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7950","reported_seq":8073,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:07.423295+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:07.423295+0000","last_peered":"2024-09-15T21:21:07.423295+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:07.423295+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":7950,"log_dups_size":0,"ondisk_log_size":7950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546816,"num_objects":7946,"num_object_clones":0,"num_object_copies":15892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5597,"num_write_kb":22388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4581},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8051","reported_seq":10934,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:07.426034+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:07.426034+0000","last_peered":"2024-09-15T21:21:07.426034+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:07.426034+0000","last_undegraded":"2024-09-15T21:21:07.426034+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32928361,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1377,"num_bytes_recovered":5640192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5947},{"shards":"0,2","objects":2093}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524608105,"num_objects":128079,"num_object_clones":0,"num_object_copies":256158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":2093,"num_objects_unfound":0,"num_objects_dirty":128079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99574,"num_write_kb":398296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4770,"num_bytes_recovered":19533824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068457984,"data_stored":1068374488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128160,"ondisk_log_size":128160,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953519,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545996,"kb_used_data":365756,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93825844,"statfs":{"total":96636764160,"available":96077664256,"internally_reserved":0,"allocated":374534144,"data_stored":374268299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707292,"kb_used_data":436876,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93664548,"statfs":{"total":96636764160,"available":95912497152,"internally_reserved":0,"allocated":447361024,"data_stored":447095179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":276885169},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,2,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451596,"kb_used_data":239420,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93920244,"statfs":{"total":96636764160,"available":96174329856,"internally_reserved":0,"allocated":245166080,"data_stored":244827176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447877120,"data_stored":447873641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245596160,"data_stored":245519622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374984704,"data_stored":374981225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:14.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 201 20 131122 131102 2.54698 1.87109 0.0225665 0.030666 2024-09-15T21:21:15.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 202 20 131608 131588 2.54377 1.89844 0.0147278 0.0307033 2024-09-15T21:21:16.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 203 20 132029 132009 2.53934 1.64453 0.0153085 0.0307581 2024-09-15T21:21:17.300 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:17.536 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 204 20 132521 132501 2.53631 1.92188 0.0530977 0.0307949 2024-09-15T21:21:18.255 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:18.256 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:18.325 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-15T21:21:16.378452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534155900,"num_objects":130412,"num_object_clones":0,"num_object_copies":260824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1948,"num_objects_unfound":0,"num_objects_dirty":130412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101907,"num_write_kb":407622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4916,"num_bytes_recovered":20127763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130494,"ondisk_log_size":130494,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736624,"kb_used_data":1061312,"kb_used_omap":12,"kb_used_meta":675251,"kb_avail":281378896,"statfs":{"total":289910292480,"available":288131989504,"internally_reserved":0,"allocated":1086783488,"data_stored":1085912894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":691457034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,10,12,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19521536,"num_objects":4766,"num_object_clones":0,"num_object_copies":9532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":4766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4766,"num_write_kb":19064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006014"},"pg_stats":[{"pgid":"2.f","version":"46'8074","reported_seq":8149,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.517309+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:12.517309+0000","last_peered":"2024-09-15T21:21:12.517309+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:12.517309+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8074,"log_dups_size":0,"ondisk_log_size":8074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8074,"num_write_kb":32296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8227","reported_seq":8323,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.062929+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:14.062929+0000","last_peered":"2024-09-15T21:21:14.062929+0000","last_clean":"2024-09-15T21:21:14.062929+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:14.062929+0000","last_undegraded":"2024-09-15T21:21:14.062929+0000","last_fullsized":"2024-09-15T21:21:14.062929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6953,"num_write_kb":27812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8230","reported_seq":15145,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.417976+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:12.417976+0000","last_peered":"2024-09-15T21:21:12.417976+0000","last_clean":"2024-09-15T21:21:12.417976+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:12.417976+0000","last_undegraded":"2024-09-15T21:21:12.417976+0000","last_fullsized":"2024-09-15T21:21:12.417976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8230,"log_dups_size":0,"ondisk_log_size":8230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6953,"num_write_kb":27812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8229","reported_seq":8297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.098541+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:14.098541+0000","last_peered":"2024-09-15T21:21:14.098541+0000","last_clean":"2024-09-15T21:21:14.098541+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:14.098541+0000","last_undegraded":"2024-09-15T21:21:14.098541+0000","last_fullsized":"2024-09-15T21:21:14.098541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8229,"log_dups_size":0,"ondisk_log_size":8229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8278","reported_seq":8396,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.416428+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:12.416428+0000","last_peered":"2024-09-15T21:21:12.416428+0000","last_clean":"2024-09-15T21:21:12.416428+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:12.416428+0000","last_undegraded":"2024-09-15T21:21:12.416428+0000","last_fullsized":"2024-09-15T21:21:12.416428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8278,"log_dups_size":0,"ondisk_log_size":8278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":24544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8115","reported_seq":8244,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.393633+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:12.393633+0000","last_peered":"2024-09-15T21:21:12.393633+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:12.393633+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8115,"log_dups_size":0,"ondisk_log_size":8115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4739},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8342","reported_seq":8465,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.418450+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:12.418450+0000","last_peered":"2024-09-15T21:21:12.418450+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:12.418450+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8342,"log_dups_size":0,"ondisk_log_size":8342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5978,"num_write_kb":23912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4865},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8075","reported_seq":8194,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.395306+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:12.395306+0000","last_peered":"2024-09-15T21:21:12.395306+0000","last_clean":"2024-09-15T21:21:12.395306+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:12.395306+0000","last_undegraded":"2024-09-15T21:21:12.395306+0000","last_fullsized":"2024-09-15T21:21:12.395306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8075,"log_dups_size":0,"ondisk_log_size":8075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067008,"num_objects":8073,"num_object_clones":0,"num_object_copies":16146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5860,"num_write_kb":23440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8097","reported_seq":8221,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:13.999458+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:13.999458+0000","last_peered":"2024-09-15T21:21:13.999458+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:13.999458+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8097,"log_dups_size":0,"ondisk_log_size":8097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33136640,"num_objects":8090,"num_object_clones":0,"num_object_copies":16180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7974","reported_seq":8095,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.479295+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:12.479295+0000","last_peered":"2024-09-15T21:21:12.479295+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:12.479295+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":7974,"log_dups_size":0,"ondisk_log_size":7974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5500,"num_write_kb":22000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4625},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8023","reported_seq":8132,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.100249+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:14.100249+0000","last_peered":"2024-09-15T21:21:14.100249+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:14.100249+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8023,"log_dups_size":0,"ondisk_log_size":8023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32841728,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6574,"num_write_kb":26296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4752}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8216","reported_seq":8330,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.104010+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:14.104010+0000","last_peered":"2024-09-15T21:21:14.104010+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:14.104010+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8216,"log_dups_size":0,"ondisk_log_size":8216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8212","reported_seq":8340,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.488703+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:12.488703+0000","last_peered":"2024-09-15T21:21:12.488703+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:12.488703+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5519,"num_write_kb":22076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4777},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8111","reported_seq":8240,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.475192+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:12.475192+0000","last_peered":"2024-09-15T21:21:12.475192+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:12.475192+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8111,"log_dups_size":0,"ondisk_log_size":8111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5696,"num_write_kb":22784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4724},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8075","reported_seq":8198,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:12.448880+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:12.448880+0000","last_peered":"2024-09-15T21:21:12.448880+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:12.448880+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8075,"log_dups_size":0,"ondisk_log_size":8075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5722,"num_write_kb":22888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4706},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8213","reported_seq":11384,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:12.480160+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:12.480160+0000","last_peered":"2024-09-15T21:21:12.480160+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:12.480160+0000","last_undegraded":"2024-09-15T21:21:12.480160+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33591913,"num_objects":8202,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1948,"num_objects_unfound":0,"num_objects_dirty":8202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5873,"num_write_kb":23492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1522,"num_bytes_recovered":6234112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6254},{"shards":"0,2","objects":1948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534155881,"num_objects":130410,"num_object_clones":0,"num_object_copies":260820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1948,"num_objects_unfound":0,"num_objects_dirty":130410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101905,"num_write_kb":407620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4915,"num_bytes_recovered":20127744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1088077824,"data_stored":1087994328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130491,"ondisk_log_size":130491,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953520,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554852,"kb_used_data":370452,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93816988,"statfs":{"total":96636764160,"available":96068595712,"internally_reserved":0,"allocated":379342848,"data_stored":379077003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718504,"kb_used_data":443928,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93653336,"statfs":{"total":96636764160,"available":95901016064,"internally_reserved":0,"allocated":454582272,"data_stored":454316427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":281145009},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463268,"kb_used_data":246932,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93908572,"statfs":{"total":96636764160,"available":96162377728,"internally_reserved":0,"allocated":252858368,"data_stored":252519464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":221507244},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455127040,"data_stored":455123561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253321216,"data_stored":253244678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":379629568,"data_stored":379626089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:18.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:18.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 205 20 133001 132981 2.53309 1.875 0.0298459 0.0308336 2024-09-15T21:21:19.137 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:19.137 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:19.176 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-15T21:21:18.378876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540820092,"num_objects":132039,"num_object_clones":0,"num_object_copies":264078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":132039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103534,"num_write_kb":414130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5057,"num_bytes_recovered":20705299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132121,"ondisk_log_size":132121,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769216,"kb_used_data":1075852,"kb_used_omap":12,"kb_used_meta":693299,"kb_avail":281346304,"statfs":{"total":289910292480,"available":288098615296,"internally_reserved":0,"allocated":1101672448,"data_stored":1100801854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":709938186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,9,12,12,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26185728,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-431,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6393,"num_write_kb":25572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":1765376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005937"},"pg_stats":[{"pgid":"2.f","version":"46'8229","reported_seq":8305,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.526370+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:17.526370+0000","last_peered":"2024-09-15T21:21:17.526370+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:17.526370+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8229,"log_dups_size":0,"ondisk_log_size":8229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8227","reported_seq":8323,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.062929+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:14.062929+0000","last_peered":"2024-09-15T21:21:14.062929+0000","last_clean":"2024-09-15T21:21:14.062929+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:14.062929+0000","last_undegraded":"2024-09-15T21:21:14.062929+0000","last_fullsized":"2024-09-15T21:21:14.062929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6953,"num_write_kb":27812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8374","reported_seq":15289,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.497761+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:17.497761+0000","last_peered":"2024-09-15T21:21:17.497761+0000","last_clean":"2024-09-15T21:21:17.497761+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:17.497761+0000","last_undegraded":"2024-09-15T21:21:17.497761+0000","last_fullsized":"2024-09-15T21:21:17.497761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8229","reported_seq":8297,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.098541+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:14.098541+0000","last_peered":"2024-09-15T21:21:14.098541+0000","last_clean":"2024-09-15T21:21:14.098541+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:14.098541+0000","last_undegraded":"2024-09-15T21:21:14.098541+0000","last_fullsized":"2024-09-15T21:21:14.098541+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8229,"log_dups_size":0,"ondisk_log_size":8229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8406","reported_seq":8524,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.464465+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:17.464465+0000","last_peered":"2024-09-15T21:21:17.464465+0000","last_clean":"2024-09-15T21:21:17.464465+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:17.464465+0000","last_undegraded":"2024-09-15T21:21:17.464465+0000","last_fullsized":"2024-09-15T21:21:17.464465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8406,"log_dups_size":0,"ondisk_log_size":8406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6264,"num_write_kb":25056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8252","reported_seq":8379,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.496637+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:17.496637+0000","last_peered":"2024-09-15T21:21:17.496637+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:17.496637+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5677,"num_write_kb":22708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4876},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8479","reported_seq":8602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.507902+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:17.507902+0000","last_peered":"2024-09-15T21:21:17.507902+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:17.507902+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8479,"log_dups_size":0,"ondisk_log_size":8479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6115,"num_write_kb":24460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5002},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8236","reported_seq":8355,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.544193+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:17.544193+0000","last_peered":"2024-09-15T21:21:17.544193+0000","last_clean":"2024-09-15T21:21:17.544193+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:17.544193+0000","last_undegraded":"2024-09-15T21:21:17.544193+0000","last_fullsized":"2024-09-15T21:21:17.544193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8236,"log_dups_size":0,"ondisk_log_size":8236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6021,"num_write_kb":24084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8097","reported_seq":8221,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:13.999458+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:13.999458+0000","last_peered":"2024-09-15T21:21:13.999458+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:13.999458+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8097,"log_dups_size":0,"ondisk_log_size":8097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33136640,"num_objects":8090,"num_object_clones":0,"num_object_copies":16180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8114","reported_seq":8236,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.487572+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:17.487572+0000","last_peered":"2024-09-15T21:21:17.487572+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:17.487572+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8114,"log_dups_size":0,"ondisk_log_size":8114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5640,"num_write_kb":22560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4765},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8023","reported_seq":8132,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.100249+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:14.100249+0000","last_peered":"2024-09-15T21:21:14.100249+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:14.100249+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8023,"log_dups_size":0,"ondisk_log_size":8023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32841728,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6574,"num_write_kb":26296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4752}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8216","reported_seq":8330,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.104010+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:14.104010+0000","last_peered":"2024-09-15T21:21:14.104010+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:14.104010+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8216,"log_dups_size":0,"ondisk_log_size":8216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33628160,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8377","reported_seq":8505,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.496812+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:17.496812+0000","last_peered":"2024-09-15T21:21:17.496812+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:17.496812+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5684,"num_write_kb":22736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8279","reported_seq":8408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.498248+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:17.498248+0000","last_peered":"2024-09-15T21:21:17.498248+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:17.498248+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8279,"log_dups_size":0,"ondisk_log_size":8279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8225","reported_seq":8348,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.526619+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:17.526619+0000","last_peered":"2024-09-15T21:21:17.526619+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:17.526619+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8225,"log_dups_size":0,"ondisk_log_size":8225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5872,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8355","reported_seq":11810,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:17.549991+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:17.549991+0000","last_peered":"2024-09-15T21:21:17.549991+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:17.549991+0000","last_undegraded":"2024-09-15T21:21:17.549991+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34173545,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6015,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1663,"num_bytes_recovered":6811648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6537},{"shards":"0,2","objects":1807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540820073,"num_objects":132037,"num_object_clones":0,"num_object_copies":264074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":132037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103532,"num_write_kb":414128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5056,"num_bytes_recovered":20705280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1103233024,"data_stored":1103149528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132118,"ondisk_log_size":132118,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953520,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554852,"kb_used_data":370452,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93816988,"statfs":{"total":96636764160,"available":96068595712,"internally_reserved":0,"allocated":379342848,"data_stored":379077003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149748,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735284,"kb_used_data":450976,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93636556,"statfs":{"total":96636764160,"available":95883833344,"internally_reserved":0,"allocated":461799424,"data_stored":461533579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":291106481},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,10,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626466,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479080,"kb_used_data":254424,"kb_used_omap":4,"kb_used_meta":224635,"kb_avail":93892760,"statfs":{"total":96636764160,"available":96146186240,"internally_reserved":0,"allocated":260530176,"data_stored":260191272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":230026924},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462475264,"data_stored":462471785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261128192,"data_stored":261051654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":379629568,"data_stored":379626089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:19.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8229", 'reported_seq': 8305, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.526370+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:17.526370+0000', 'last_peered': '2024-09-15T21:21:17.526370+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:17.526370+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 8229, 'log_dups_size': 0, 'ondisk_log_size': 8229, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33705984, 'num_objects': 8229, 'num_object_clones': 0, 'num_object_copies': 16458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8229, 'num_write_kb': 32916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4862}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8252", 'reported_seq': 8379, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.496637+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:17.496637+0000', 'last_peered': '2024-09-15T21:21:17.496637+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:17.496637+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 8252, 'log_dups_size': 0, 'ondisk_log_size': 8252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33779712, 'num_objects': 8247, 'num_object_clones': 0, 'num_object_copies': 16494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5677, 'num_write_kb': 22708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4876}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8479", 'reported_seq': 8602, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.507902+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:17.507902+0000', 'last_peered': '2024-09-15T21:21:17.507902+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:17.507902+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8479, 'log_dups_size': 0, 'ondisk_log_size': 8479, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34705408, 'num_objects': 8473, 'num_object_clones': 0, 'num_object_copies': 16946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6115, 'num_write_kb': 24460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5002}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:19.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8097", 'reported_seq': 8221, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:13.999458+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:13.999458+0000', 'last_peered': '2024-09-15T21:21:13.999458+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:13.999458+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8097, 'log_dups_size': 0, 'ondisk_log_size': 8097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33136640, 'num_objects': 8090, 'num_object_clones': 0, 'num_object_copies': 16180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6683, 'num_write_kb': 26732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 4752}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8114", 'reported_seq': 8236, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.487572+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:17.487572+0000', 'last_peered': '2024-09-15T21:21:17.487572+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:17.487572+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8114, 'log_dups_size': 0, 'ondisk_log_size': 8114, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33202176, 'num_objects': 8106, 'num_object_clones': 0, 'num_object_copies': 16212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5640, 'num_write_kb': 22560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4765}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8023", 'reported_seq': 8132, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.100249+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:14.100249+0000', 'last_peered': '2024-09-15T21:21:14.100249+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:14.100249+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8023, 'log_dups_size': 0, 'ondisk_log_size': 8023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32841728, 'num_objects': 8018, 'num_object_clones': 0, 'num_object_copies': 16036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6574, 'num_write_kb': 26296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 4752}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8216", 'reported_seq': 8330, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.104010+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:14.104010+0000', 'last_peered': '2024-09-15T21:21:14.104010+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:14.104010+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8216, 'log_dups_size': 0, 'ondisk_log_size': 8216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33628160, 'num_objects': 8210, 'num_object_clones': 0, 'num_object_copies': 16420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6615, 'num_write_kb': 26460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4860}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8377", 'reported_seq': 8505, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.496812+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:17.496812+0000', 'last_peered': '2024-09-15T21:21:17.496812+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:17.496812+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8377, 'log_dups_size': 0, 'ondisk_log_size': 8377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34304000, 'num_objects': 8375, 'num_object_clones': 0, 'num_object_copies': 16750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5684, 'num_write_kb': 22736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4942}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:19.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8279", 'reported_seq': 8408, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.498248+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:17.498248+0000', 'last_peered': '2024-09-15T21:21:17.498248+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:17.498248+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 8279, 'log_dups_size': 0, 'ondisk_log_size': 8279, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33878016, 'num_objects': 8271, 'num_object_clones': 0, 'num_object_copies': 16542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5864, 'num_write_kb': 23456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4892}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:19.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8225", 'reported_seq': 8348, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.526619+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:17.526619+0000', 'last_peered': '2024-09-15T21:21:17.526619+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:17.526619+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 8225, 'log_dups_size': 0, 'ondisk_log_size': 8225, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33673216, 'num_objects': 8221, 'num_object_clones': 0, 'num_object_copies': 16442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5872, 'num_write_kb': 23488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4856}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:19.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8355", 'reported_seq': 11810, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:17.549991+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:17.549991+0000', 'last_peered': '2024-09-15T21:21:17.549991+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:17.549991+0000', 'last_undegraded': '2024-09-15T21:21:17.549991+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8355, 'log_dups_size': 0, 'ondisk_log_size': 8355, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34173545, 'num_objects': 8344, 'num_object_clones': 0, 'num_object_copies': 16688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1807, 'num_objects_unfound': 0, 'num_objects_dirty': 8344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6015, 'num_write_kb': 24060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1663, 'num_bytes_recovered': 6811648, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6537}, {'shards': '0,2', 'objects': 1807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:19.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:19.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 206 20 133474 133454 2.52976 1.84766 0.108907 0.030873 2024-09-15T21:21:20.439 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:20.440 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/264078 objects degraded (12.761%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":293,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132039,"data_bytes":540820092,"bytes_used":1811677184,"bytes_avail":288098615296,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":264078,"degraded_ratio":0.12761002431099902,"misplaced_objects":1807,"misplaced_total":264078,"misplaced_ratio":0.0068426752701853241,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":147041,"recovering_keys_per_sec":0,"num_objects_recovered":431,"num_bytes_recovered":1765376,"num_keys_recovered":0,"write_bytes_sec":2181064,"read_op_per_sec":0,"write_op_per_sec":532},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:20.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:20.493 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:20.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 207 20 133943 133923 2.52639 1.83203 0.0233479 0.0309157 2024-09-15T21:21:21.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:21.026 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:21.068 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-15T21:21:20.379362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543707772,"num_objects":132744,"num_object_clones":0,"num_object_copies":265488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":132744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104239,"num_write_kb":416950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5057,"num_bytes_recovered":20705299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132826,"ondisk_log_size":132826,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1778032,"kb_used_data":1080508,"kb_used_omap":12,"kb_used_meta":697459,"kb_avail":281337488,"statfs":{"total":289910292480,"available":288089587712,"internally_reserved":0,"allocated":1106440192,"data_stored":1105569598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":714198026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,12,9,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22126592,"num_objects":5402,"num_object_clones":0,"num_object_copies":10804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":5402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5402,"num_write_kb":21608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005980"},"pg_stats":[{"pgid":"2.f","version":"46'8229","reported_seq":8305,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.526370+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:17.526370+0000","last_peered":"2024-09-15T21:21:17.526370+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:17.526370+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8229,"log_dups_size":0,"ondisk_log_size":8229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":32916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8374","reported_seq":8470,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:19.114429+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:19.114429+0000","last_peered":"2024-09-15T21:21:19.114429+0000","last_clean":"2024-09-15T21:21:19.114429+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:19.114429+0000","last_undegraded":"2024-09-15T21:21:19.114429+0000","last_fullsized":"2024-09-15T21:21:19.114429+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7100,"num_write_kb":28400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8374","reported_seq":15289,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.497761+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:17.497761+0000","last_peered":"2024-09-15T21:21:17.497761+0000","last_clean":"2024-09-15T21:21:17.497761+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:17.497761+0000","last_undegraded":"2024-09-15T21:21:17.497761+0000","last_fullsized":"2024-09-15T21:21:17.497761+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7097,"num_write_kb":28388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8385","reported_seq":8454,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:19.009367+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:19.009367+0000","last_peered":"2024-09-15T21:21:19.009367+0000","last_clean":"2024-09-15T21:21:19.009367+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:19.009367+0000","last_undegraded":"2024-09-15T21:21:19.009367+0000","last_fullsized":"2024-09-15T21:21:19.009367+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8385,"log_dups_size":0,"ondisk_log_size":8385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34344960,"num_objects":8385,"num_object_clones":0,"num_object_copies":16770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":33540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8406","reported_seq":8524,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.464465+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:17.464465+0000","last_peered":"2024-09-15T21:21:17.464465+0000","last_clean":"2024-09-15T21:21:17.464465+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:17.464465+0000","last_undegraded":"2024-09-15T21:21:17.464465+0000","last_fullsized":"2024-09-15T21:21:17.464465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8406,"log_dups_size":0,"ondisk_log_size":8406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6264,"num_write_kb":25056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8252","reported_seq":8379,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.496637+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:17.496637+0000","last_peered":"2024-09-15T21:21:17.496637+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:17.496637+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5677,"num_write_kb":22708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4876},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8479","reported_seq":8602,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.507902+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:17.507902+0000","last_peered":"2024-09-15T21:21:17.507902+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:17.507902+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8479,"log_dups_size":0,"ondisk_log_size":8479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6115,"num_write_kb":24460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5002},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8236","reported_seq":8355,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.544193+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:17.544193+0000","last_peered":"2024-09-15T21:21:17.544193+0000","last_clean":"2024-09-15T21:21:17.544193+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:17.544193+0000","last_undegraded":"2024-09-15T21:21:17.544193+0000","last_fullsized":"2024-09-15T21:21:17.544193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8236,"log_dups_size":0,"ondisk_log_size":8236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6021,"num_write_kb":24084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8230","reported_seq":8354,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.040399+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:19.040399+0000","last_peered":"2024-09-15T21:21:19.040399+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:19.040399+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8230,"log_dups_size":0,"ondisk_log_size":8230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":27264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":4885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8114","reported_seq":8236,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.487572+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:17.487572+0000","last_peered":"2024-09-15T21:21:17.487572+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:17.487572+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8114,"log_dups_size":0,"ondisk_log_size":8114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5640,"num_write_kb":22560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4765},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8159","reported_seq":8268,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.013284+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:19.013284+0000","last_peered":"2024-09-15T21:21:19.013284+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:19.013284+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8159,"log_dups_size":0,"ondisk_log_size":8159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6710,"num_write_kb":26840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":4888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8349","reported_seq":8463,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.098038+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:19.098038+0000","last_peered":"2024-09-15T21:21:19.098038+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:19.098038+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8349,"log_dups_size":0,"ondisk_log_size":8349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34172928,"num_objects":8343,"num_object_clones":0,"num_object_copies":16686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8377","reported_seq":8505,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.496812+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:17.496812+0000","last_peered":"2024-09-15T21:21:17.496812+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:17.496812+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5684,"num_write_kb":22736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8279","reported_seq":8408,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.498248+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:17.498248+0000","last_peered":"2024-09-15T21:21:17.498248+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:17.498248+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8279,"log_dups_size":0,"ondisk_log_size":8279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":23456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8225","reported_seq":8348,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.526619+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:17.526619+0000","last_peered":"2024-09-15T21:21:17.526619+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:17.526619+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8225,"log_dups_size":0,"ondisk_log_size":8225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673216,"num_objects":8221,"num_object_clones":0,"num_object_copies":16442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5872,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4856},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8355","reported_seq":11810,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:17.549991+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:17.549991+0000","last_peered":"2024-09-15T21:21:17.549991+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:17.549991+0000","last_undegraded":"2024-09-15T21:21:17.549991+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34173545,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6015,"num_write_kb":24060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1663,"num_bytes_recovered":6811648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6537},{"shards":"0,2","objects":1807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543707753,"num_objects":132742,"num_object_clones":0,"num_object_copies":265484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1807,"num_objects_unfound":0,"num_objects_dirty":132742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104237,"num_write_kb":416948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5056,"num_bytes_recovered":20705280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107951616,"data_stored":1107868120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132823,"ondisk_log_size":132823,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953521,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563668,"kb_used_data":375108,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93808172,"statfs":{"total":96636764160,"available":96059568128,"internally_reserved":0,"allocated":384110592,"data_stored":383844747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":193064621},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149748,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735284,"kb_used_data":450976,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93636556,"statfs":{"total":96636764160,"available":95883833344,"internally_reserved":0,"allocated":461799424,"data_stored":461533579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":291106481},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,10,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626466,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479080,"kb_used_data":254424,"kb_used_omap":4,"kb_used_meta":224635,"kb_avail":93892760,"statfs":{"total":96636764160,"available":96146186240,"internally_reserved":0,"allocated":260530176,"data_stored":260191272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":230026924},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462475264,"data_stored":462471785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":261128192,"data_stored":261051654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":384348160,"data_stored":384344681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:21.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 208 20 134361 134341 2.52209 1.63281 0.0340336 0.0309702 2024-09-15T21:21:22.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 209 20 134869 134849 2.51952 1.98438 0.023686 0.0309992 2024-09-15T21:21:23.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 210 20 135378 135358 2.51699 1.98828 0.0151563 0.0310314 2024-09-15T21:21:24.071 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:24.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 211 20 135765 135745 2.51223 1.51172 0.0265706 0.0310903 2024-09-15T21:21:24.827 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:24.828 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:24.869 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-15T21:21:24.380306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":553415292,"num_objects":135114,"num_object_clones":0,"num_object_copies":270228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106609,"num_write_kb":426430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":21303315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135196,"ondisk_log_size":135196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1776852,"kb_used_data":1100132,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281338668,"statfs":{"total":289910292480,"available":288090796032,"internally_reserved":0,"allocated":1126535168,"data_stored":1125675011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":692898826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,8,9,8,6,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-432,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7033,"num_write_kb":28132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002815"},"pg_stats":[{"pgid":"2.f","version":"46'8395","reported_seq":8471,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.562511+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:22.562511+0000","last_peered":"2024-09-15T21:21:22.562511+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:22.562511+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8395,"log_dups_size":0,"ondisk_log_size":8395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8503","reported_seq":8599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.915631+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:23.915631+0000","last_peered":"2024-09-15T21:21:23.915631+0000","last_clean":"2024-09-15T21:21:23.915631+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:23.915631+0000","last_undegraded":"2024-09-15T21:21:23.915631+0000","last_fullsized":"2024-09-15T21:21:23.915631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8503,"log_dups_size":0,"ondisk_log_size":8503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34803712,"num_objects":8497,"num_object_clones":0,"num_object_copies":16994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8538","reported_seq":15453,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.518096+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:22.518096+0000","last_peered":"2024-09-15T21:21:22.518096+0000","last_clean":"2024-09-15T21:21:22.518096+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:22.518096+0000","last_undegraded":"2024-09-15T21:21:22.518096+0000","last_fullsized":"2024-09-15T21:21:22.518096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8538,"log_dups_size":0,"ondisk_log_size":8538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7261,"num_write_kb":29044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8529","reported_seq":8598,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:24.038594+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:24.038594+0000","last_peered":"2024-09-15T21:21:24.038594+0000","last_clean":"2024-09-15T21:21:24.038594+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:24.038594+0000","last_undegraded":"2024-09-15T21:21:24.038594+0000","last_fullsized":"2024-09-15T21:21:24.038594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8529,"log_dups_size":0,"ondisk_log_size":8529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34934784,"num_objects":8529,"num_object_clones":0,"num_object_copies":17058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":34116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8561","reported_seq":8679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.536338+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:22.536338+0000","last_peered":"2024-09-15T21:21:22.536338+0000","last_clean":"2024-09-15T21:21:22.536338+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:22.536338+0000","last_undegraded":"2024-09-15T21:21:22.536338+0000","last_fullsized":"2024-09-15T21:21:22.536338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8561,"log_dups_size":0,"ondisk_log_size":8561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037184,"num_objects":8554,"num_object_clones":0,"num_object_copies":17108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8392","reported_seq":8521,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.523749+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:22.523749+0000","last_peered":"2024-09-15T21:21:22.523749+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:22.523749+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8627","reported_seq":8748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.546529+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:22.546529+0000","last_peered":"2024-09-15T21:21:22.546529+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:22.546529+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6263,"num_write_kb":25052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8394","reported_seq":8513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.472857+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:22.472857+0000","last_peered":"2024-09-15T21:21:22.472857+0000","last_clean":"2024-09-15T21:21:22.472857+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:22.472857+0000","last_undegraded":"2024-09-15T21:21:22.472857+0000","last_fullsized":"2024-09-15T21:21:22.472857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8394,"log_dups_size":0,"ondisk_log_size":8394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6179,"num_write_kb":24716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8382","reported_seq":8506,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.038378+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:24.038378+0000","last_peered":"2024-09-15T21:21:24.038378+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:24.038378+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8382,"log_dups_size":0,"ondisk_log_size":8382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8244","reported_seq":8366,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.458674+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:22.458674+0000","last_peered":"2024-09-15T21:21:22.458674+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:22.458674+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33734656,"num_objects":8236,"num_object_clones":0,"num_object_copies":16472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5770,"num_write_kb":23080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4895},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8314","reported_seq":8421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.083649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:24.083649+0000","last_peered":"2024-09-15T21:21:24.083649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:24.083649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34033664,"num_objects":8309,"num_object_clones":0,"num_object_copies":16618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5043}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8499","reported_seq":8613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.115236+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:24.115236+0000","last_peered":"2024-09-15T21:21:24.115236+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:24.115236+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34787328,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8520","reported_seq":8648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.557588+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:22.557588+0000","last_peered":"2024-09-15T21:21:22.557588+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:22.557588+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34889728,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5827,"num_write_kb":23308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8419","reported_seq":8548,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.537468+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:22.537468+0000","last_peered":"2024-09-15T21:21:22.537468+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:22.537468+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8377","reported_seq":8500,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.576419+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:22.576419+0000","last_peered":"2024-09-15T21:21:22.576419+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:22.576419+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6024,"num_write_kb":24096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8499","reported_seq":12251,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:22.550010+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:22.550010+0000","last_peered":"2024-09-15T21:21:22.550010+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:22.550010+0000","last_undegraded":"2024-09-15T21:21:22.550010+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34763369,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6159,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6827},{"shards":"0,2","objects":1661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":553415273,"num_objects":135112,"num_object_clones":0,"num_object_copies":270224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106607,"num_write_kb":426428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5205,"num_bytes_recovered":21303296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1127837696,"data_stored":1127764637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135193,"ondisk_log_size":135193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556264,"kb_used_data":380184,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93815576,"statfs":{"total":96636764160,"available":96067149824,"internally_reserved":0,"allocated":389308416,"data_stored":389042571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,7,1,2,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746464,"kb_used_data":458000,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93625376,"statfs":{"total":96636764160,"available":95872385024,"internally_reserved":0,"allocated":468992000,"data_stored":468726155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":295366321},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474124,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93897716,"statfs":{"total":96636764160,"available":96151261184,"internally_reserved":0,"allocated":268234752,"data_stored":267906285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469659648,"data_stored":469656169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268996608,"data_stored":268930507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389181440,"data_stored":389177961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:24.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:25.537 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 212 20 136224 136204 2.50883 1.79297 0.0237075 0.0311315 2024-09-15T21:21:25.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:25.618 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:25.657 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-15T21:21:24.380306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":553415292,"num_objects":135114,"num_object_clones":0,"num_object_copies":270228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106609,"num_write_kb":426430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":21303315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135196,"ondisk_log_size":135196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1776852,"kb_used_data":1100132,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281338668,"statfs":{"total":289910292480,"available":288090796032,"internally_reserved":0,"allocated":1126535168,"data_stored":1125675011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":692898826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,8,9,8,6,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-432,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7033,"num_write_kb":28132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002815"},"pg_stats":[{"pgid":"2.f","version":"46'8395","reported_seq":8471,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.562511+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:22.562511+0000","last_peered":"2024-09-15T21:21:22.562511+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:22.562511+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8395,"log_dups_size":0,"ondisk_log_size":8395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8503","reported_seq":8599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.915631+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:23.915631+0000","last_peered":"2024-09-15T21:21:23.915631+0000","last_clean":"2024-09-15T21:21:23.915631+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:23.915631+0000","last_undegraded":"2024-09-15T21:21:23.915631+0000","last_fullsized":"2024-09-15T21:21:23.915631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8503,"log_dups_size":0,"ondisk_log_size":8503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34803712,"num_objects":8497,"num_object_clones":0,"num_object_copies":16994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8538","reported_seq":15453,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.518096+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:22.518096+0000","last_peered":"2024-09-15T21:21:22.518096+0000","last_clean":"2024-09-15T21:21:22.518096+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:22.518096+0000","last_undegraded":"2024-09-15T21:21:22.518096+0000","last_fullsized":"2024-09-15T21:21:22.518096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8538,"log_dups_size":0,"ondisk_log_size":8538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7261,"num_write_kb":29044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8529","reported_seq":8598,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:24.038594+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:24.038594+0000","last_peered":"2024-09-15T21:21:24.038594+0000","last_clean":"2024-09-15T21:21:24.038594+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:24.038594+0000","last_undegraded":"2024-09-15T21:21:24.038594+0000","last_fullsized":"2024-09-15T21:21:24.038594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8529,"log_dups_size":0,"ondisk_log_size":8529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34934784,"num_objects":8529,"num_object_clones":0,"num_object_copies":17058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":34116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8561","reported_seq":8679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.536338+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:22.536338+0000","last_peered":"2024-09-15T21:21:22.536338+0000","last_clean":"2024-09-15T21:21:22.536338+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:22.536338+0000","last_undegraded":"2024-09-15T21:21:22.536338+0000","last_fullsized":"2024-09-15T21:21:22.536338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8561,"log_dups_size":0,"ondisk_log_size":8561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037184,"num_objects":8554,"num_object_clones":0,"num_object_copies":17108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8392","reported_seq":8521,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.523749+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:22.523749+0000","last_peered":"2024-09-15T21:21:22.523749+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:22.523749+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8627","reported_seq":8748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.546529+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:22.546529+0000","last_peered":"2024-09-15T21:21:22.546529+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:22.546529+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6263,"num_write_kb":25052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8394","reported_seq":8513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.472857+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:22.472857+0000","last_peered":"2024-09-15T21:21:22.472857+0000","last_clean":"2024-09-15T21:21:22.472857+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:22.472857+0000","last_undegraded":"2024-09-15T21:21:22.472857+0000","last_fullsized":"2024-09-15T21:21:22.472857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8394,"log_dups_size":0,"ondisk_log_size":8394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6179,"num_write_kb":24716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8382","reported_seq":8506,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.038378+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:24.038378+0000","last_peered":"2024-09-15T21:21:24.038378+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:24.038378+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8382,"log_dups_size":0,"ondisk_log_size":8382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8244","reported_seq":8366,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.458674+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:22.458674+0000","last_peered":"2024-09-15T21:21:22.458674+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:22.458674+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33734656,"num_objects":8236,"num_object_clones":0,"num_object_copies":16472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5770,"num_write_kb":23080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4895},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8314","reported_seq":8421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.083649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:24.083649+0000","last_peered":"2024-09-15T21:21:24.083649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:24.083649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34033664,"num_objects":8309,"num_object_clones":0,"num_object_copies":16618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5043}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8499","reported_seq":8613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.115236+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:24.115236+0000","last_peered":"2024-09-15T21:21:24.115236+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:24.115236+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34787328,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8520","reported_seq":8648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.557588+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:22.557588+0000","last_peered":"2024-09-15T21:21:22.557588+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:22.557588+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34889728,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5827,"num_write_kb":23308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8419","reported_seq":8548,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.537468+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:22.537468+0000","last_peered":"2024-09-15T21:21:22.537468+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:22.537468+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8377","reported_seq":8500,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.576419+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:22.576419+0000","last_peered":"2024-09-15T21:21:22.576419+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:22.576419+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6024,"num_write_kb":24096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8499","reported_seq":12251,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:22.550010+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:22.550010+0000","last_peered":"2024-09-15T21:21:22.550010+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:22.550010+0000","last_undegraded":"2024-09-15T21:21:22.550010+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34763369,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6159,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6827},{"shards":"0,2","objects":1661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":553415273,"num_objects":135112,"num_object_clones":0,"num_object_copies":270224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106607,"num_write_kb":426428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5205,"num_bytes_recovered":21303296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1127837696,"data_stored":1127764637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135193,"ondisk_log_size":135193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556264,"kb_used_data":380184,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93815576,"statfs":{"total":96636764160,"available":96067149824,"internally_reserved":0,"allocated":389308416,"data_stored":389042571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,7,1,2,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746464,"kb_used_data":458000,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93625376,"statfs":{"total":96636764160,"available":95872385024,"internally_reserved":0,"allocated":468992000,"data_stored":468726155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":295366321},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474124,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93897716,"statfs":{"total":96636764160,"available":96151261184,"internally_reserved":0,"allocated":268234752,"data_stored":267906285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469659648,"data_stored":469656169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268996608,"data_stored":268930507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389181440,"data_stored":389177961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8395", 'reported_seq': 8471, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.562511+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:22.562511+0000', 'last_peered': '2024-09-15T21:21:22.562511+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:22.562511+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 8395, 'log_dups_size': 0, 'ondisk_log_size': 8395, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34385920, 'num_objects': 8395, 'num_object_clones': 0, 'num_object_copies': 16790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8395, 'num_write_kb': 33580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5028}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8392", 'reported_seq': 8521, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.523749+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:22.523749+0000', 'last_peered': '2024-09-15T21:21:22.523749+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:22.523749+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 8392, 'log_dups_size': 0, 'ondisk_log_size': 8392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34353152, 'num_objects': 8387, 'num_object_clones': 0, 'num_object_copies': 16774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5817, 'num_write_kb': 23268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5016}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8627", 'reported_seq': 8748, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.546529+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:22.546529+0000', 'last_peered': '2024-09-15T21:21:22.546529+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:22.546529+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8627, 'log_dups_size': 0, 'ondisk_log_size': 8627, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35311616, 'num_objects': 8621, 'num_object_clones': 0, 'num_object_copies': 17242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6263, 'num_write_kb': 25052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5150}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:25.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8382", 'reported_seq': 8506, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:24.038378+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:24.038378+0000', 'last_peered': '2024-09-15T21:21:24.038378+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:24.038378+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8382, 'log_dups_size': 0, 'ondisk_log_size': 8382, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34304000, 'num_objects': 8375, 'num_object_clones': 0, 'num_object_copies': 16750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6968, 'num_write_kb': 27872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5037}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8244", 'reported_seq': 8366, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.458674+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:22.458674+0000', 'last_peered': '2024-09-15T21:21:22.458674+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:22.458674+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8244, 'log_dups_size': 0, 'ondisk_log_size': 8244, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33734656, 'num_objects': 8236, 'num_object_clones': 0, 'num_object_copies': 16472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5770, 'num_write_kb': 23080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4895}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8314", 'reported_seq': 8421, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:24.083649+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:24.083649+0000', 'last_peered': '2024-09-15T21:21:24.083649+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:24.083649+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8314, 'log_dups_size': 0, 'ondisk_log_size': 8314, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34033664, 'num_objects': 8309, 'num_object_clones': 0, 'num_object_copies': 16618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6865, 'num_write_kb': 27460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 5043}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8499", 'reported_seq': 8613, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:24.115236+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:24.115236+0000', 'last_peered': '2024-09-15T21:21:24.115236+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:24.115236+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8499, 'log_dups_size': 0, 'ondisk_log_size': 8499, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34787328, 'num_objects': 8493, 'num_object_clones': 0, 'num_object_copies': 16986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6898, 'num_write_kb': 27592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 5143}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8520", 'reported_seq': 8648, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.557588+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:22.557588+0000', 'last_peered': '2024-09-15T21:21:22.557588+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:22.557588+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8520, 'log_dups_size': 0, 'ondisk_log_size': 8520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34889728, 'num_objects': 8518, 'num_object_clones': 0, 'num_object_copies': 17036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5827, 'num_write_kb': 23308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5085}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:25.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8419", 'reported_seq': 8548, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.537468+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:22.537468+0000', 'last_peered': '2024-09-15T21:21:22.537468+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:22.537468+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 8419, 'log_dups_size': 0, 'ondisk_log_size': 8419, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34451456, 'num_objects': 8411, 'num_object_clones': 0, 'num_object_copies': 16822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6004, 'num_write_kb': 24016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5032}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:25.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8377", 'reported_seq': 8500, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:22.576419+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:22.576419+0000', 'last_peered': '2024-09-15T21:21:22.576419+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:22.576419+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 8377, 'log_dups_size': 0, 'ondisk_log_size': 8377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34295808, 'num_objects': 8373, 'num_object_clones': 0, 'num_object_copies': 16746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6024, 'num_write_kb': 24096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5008}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:25.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8499", 'reported_seq': 12251, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:22.550010+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:22.550010+0000', 'last_peered': '2024-09-15T21:21:22.550010+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:22.550010+0000', 'last_undegraded': '2024-09-15T21:21:22.550010+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8499, 'log_dups_size': 0, 'ondisk_log_size': 8499, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34763369, 'num_objects': 8488, 'num_object_clones': 0, 'num_object_copies': 16976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1661, 'num_objects_unfound': 0, 'num_objects_dirty': 8488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6159, 'num_write_kb': 24636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1812, 'num_bytes_recovered': 7409664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6827}, {'shards': '0,2', 'objects': 1661}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:25.661 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:26.301 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:26.301 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/270228 objects degraded (12.471%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135114,"data_bytes":553415292,"bytes_used":1819496448,"bytes_avail":288090796032,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":270228,"degraded_ratio":0.12470580398774368,"misplaced_objects":1661,"misplaced_total":270228,"misplaced_ratio":0.0061466613378332365,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":147421,"recovering_keys_per_sec":0,"num_objects_recovered":435,"num_bytes_recovered":1769472,"num_keys_recovered":0,"write_bytes_sec":2400034,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:26.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:26.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:26.541 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 213 20 136593 136573 2.50378 1.44141 0.0289084 0.0311937 2024-09-15T21:21:27.168 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:27.168 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:27.215 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-15T21:21:26.380820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":553415292,"num_objects":135114,"num_object_clones":0,"num_object_copies":270228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106609,"num_write_kb":426430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5206,"num_bytes_recovered":21303315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135196,"ondisk_log_size":135196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1776852,"kb_used_data":1100132,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281338668,"statfs":{"total":289910292480,"available":288090796032,"internally_reserved":0,"allocated":1126535168,"data_stored":1125675011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":692898826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,8,9,8,6,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":18808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002879"},"pg_stats":[{"pgid":"2.f","version":"46'8395","reported_seq":8471,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.562511+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:22.562511+0000","last_peered":"2024-09-15T21:21:22.562511+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:22.562511+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8395,"log_dups_size":0,"ondisk_log_size":8395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34385920,"num_objects":8395,"num_object_clones":0,"num_object_copies":16790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8503","reported_seq":8599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.915631+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:23.915631+0000","last_peered":"2024-09-15T21:21:23.915631+0000","last_clean":"2024-09-15T21:21:23.915631+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:23.915631+0000","last_undegraded":"2024-09-15T21:21:23.915631+0000","last_fullsized":"2024-09-15T21:21:23.915631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8503,"log_dups_size":0,"ondisk_log_size":8503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34803712,"num_objects":8497,"num_object_clones":0,"num_object_copies":16994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8538","reported_seq":15453,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.518096+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:22.518096+0000","last_peered":"2024-09-15T21:21:22.518096+0000","last_clean":"2024-09-15T21:21:22.518096+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:22.518096+0000","last_undegraded":"2024-09-15T21:21:22.518096+0000","last_fullsized":"2024-09-15T21:21:22.518096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8538,"log_dups_size":0,"ondisk_log_size":8538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7261,"num_write_kb":29044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8529","reported_seq":8598,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:24.038594+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:24.038594+0000","last_peered":"2024-09-15T21:21:24.038594+0000","last_clean":"2024-09-15T21:21:24.038594+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:24.038594+0000","last_undegraded":"2024-09-15T21:21:24.038594+0000","last_fullsized":"2024-09-15T21:21:24.038594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8529,"log_dups_size":0,"ondisk_log_size":8529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34934784,"num_objects":8529,"num_object_clones":0,"num_object_copies":17058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":34116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8561","reported_seq":8679,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.536338+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:22.536338+0000","last_peered":"2024-09-15T21:21:22.536338+0000","last_clean":"2024-09-15T21:21:22.536338+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:22.536338+0000","last_undegraded":"2024-09-15T21:21:22.536338+0000","last_fullsized":"2024-09-15T21:21:22.536338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8561,"log_dups_size":0,"ondisk_log_size":8561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35037184,"num_objects":8554,"num_object_clones":0,"num_object_copies":17108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8392","reported_seq":8521,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.523749+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:22.523749+0000","last_peered":"2024-09-15T21:21:22.523749+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:22.523749+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5016},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8627","reported_seq":8748,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.546529+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:22.546529+0000","last_peered":"2024-09-15T21:21:22.546529+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:22.546529+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8627,"log_dups_size":0,"ondisk_log_size":8627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6263,"num_write_kb":25052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5150},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8394","reported_seq":8513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:22.472857+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:22.472857+0000","last_peered":"2024-09-15T21:21:22.472857+0000","last_clean":"2024-09-15T21:21:22.472857+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:22.472857+0000","last_undegraded":"2024-09-15T21:21:22.472857+0000","last_fullsized":"2024-09-15T21:21:22.472857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8394,"log_dups_size":0,"ondisk_log_size":8394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6179,"num_write_kb":24716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8382","reported_seq":8506,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.038378+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:24.038378+0000","last_peered":"2024-09-15T21:21:24.038378+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:24.038378+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8382,"log_dups_size":0,"ondisk_log_size":8382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8244","reported_seq":8366,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.458674+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:22.458674+0000","last_peered":"2024-09-15T21:21:22.458674+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:22.458674+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33734656,"num_objects":8236,"num_object_clones":0,"num_object_copies":16472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5770,"num_write_kb":23080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4895},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8314","reported_seq":8421,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.083649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:24.083649+0000","last_peered":"2024-09-15T21:21:24.083649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:24.083649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8314,"log_dups_size":0,"ondisk_log_size":8314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34033664,"num_objects":8309,"num_object_clones":0,"num_object_copies":16618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6865,"num_write_kb":27460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5043}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8499","reported_seq":8613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.115236+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:24.115236+0000","last_peered":"2024-09-15T21:21:24.115236+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:24.115236+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34787328,"num_objects":8493,"num_object_clones":0,"num_object_copies":16986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8520","reported_seq":8648,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.557588+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:22.557588+0000","last_peered":"2024-09-15T21:21:22.557588+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:22.557588+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34889728,"num_objects":8518,"num_object_clones":0,"num_object_copies":17036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5827,"num_write_kb":23308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8419","reported_seq":8548,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.537468+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:22.537468+0000","last_peered":"2024-09-15T21:21:22.537468+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:22.537468+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8419,"log_dups_size":0,"ondisk_log_size":8419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34451456,"num_objects":8411,"num_object_clones":0,"num_object_copies":16822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8377","reported_seq":8500,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.576419+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:22.576419+0000","last_peered":"2024-09-15T21:21:22.576419+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:22.576419+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6024,"num_write_kb":24096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8499","reported_seq":12251,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:22.550010+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:22.550010+0000","last_peered":"2024-09-15T21:21:22.550010+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:22.550010+0000","last_undegraded":"2024-09-15T21:21:22.550010+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8499,"log_dups_size":0,"ondisk_log_size":8499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34763369,"num_objects":8488,"num_object_clones":0,"num_object_copies":16976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":8488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6159,"num_write_kb":24636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6827},{"shards":"0,2","objects":1661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":553415273,"num_objects":135112,"num_object_clones":0,"num_object_copies":270224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1661,"num_objects_unfound":0,"num_objects_dirty":135112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106607,"num_write_kb":426428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5205,"num_bytes_recovered":21303296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1127837696,"data_stored":1127764637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135193,"ondisk_log_size":135193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556264,"kb_used_data":380184,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93815576,"statfs":{"total":96636764160,"available":96067149824,"internally_reserved":0,"allocated":389308416,"data_stored":389042571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,8,7,1,2,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149749,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746464,"kb_used_data":458000,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93625376,"statfs":{"total":96636764160,"available":95872385024,"internally_reserved":0,"allocated":468992000,"data_stored":468726155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":295366321},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626467,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474124,"kb_used_data":261948,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93897716,"statfs":{"total":96636764160,"available":96151261184,"internally_reserved":0,"allocated":268234752,"data_stored":267906285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":217247404},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469659648,"data_stored":469656169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":268996608,"data_stored":268930507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389181440,"data_stored":389177961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:27.541 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 214 20 137060 137040 2.50061 1.82422 0.0139861 0.0312332 2024-09-15T21:21:28.541 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 215 20 137554 137534 2.49795 1.92969 0.0251727 0.0312683 2024-09-15T21:21:29.544 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 216 20 137974 137954 2.49394 1.64062 0.228176 0.0313053 2024-09-15T21:21:30.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:30.544 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 217 20 138364 138344 2.48947 1.52344 0.0275506 0.0313736 2024-09-15T21:21:30.754 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:30.783 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-15T21:21:30.388435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":562578044,"num_objects":137351,"num_object_clones":0,"num_object_copies":274702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108846,"num_write_kb":435378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5381,"num_bytes_recovered":22020115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137433,"ondisk_log_size":137433,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815772,"kb_used_data":1118252,"kb_used_omap":12,"kb_used_meta":697459,"kb_avail":281299748,"statfs":{"total":289910292480,"available":288050941952,"internally_reserved":0,"allocated":1145090048,"data_stored":1144229891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":714198026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,4,6,8,11,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21757952,"num_objects":5312,"num_object_clones":0,"num_object_copies":10624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":5312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5312,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009559"},"pg_stats":[{"pgid":"2.f","version":"46'8527","reported_seq":8603,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.591347+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:27.591347+0000","last_peered":"2024-09-15T21:21:27.591347+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:27.591347+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34926592,"num_objects":8527,"num_object_clones":0,"num_object_copies":17054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8527,"num_write_kb":34108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8654","reported_seq":8750,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.116648+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:29.116648+0000","last_peered":"2024-09-15T21:21:29.116648+0000","last_clean":"2024-09-15T21:21:29.116648+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:29.116648+0000","last_undegraded":"2024-09-15T21:21:29.116648+0000","last_fullsized":"2024-09-15T21:21:29.116648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35422208,"num_objects":8648,"num_object_clones":0,"num_object_copies":17296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":29520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8684","reported_seq":15599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.547674+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:27.547674+0000","last_peered":"2024-09-15T21:21:27.547674+0000","last_clean":"2024-09-15T21:21:27.547674+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:27.547674+0000","last_undegraded":"2024-09-15T21:21:27.547674+0000","last_fullsized":"2024-09-15T21:21:27.547674+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8684,"log_dups_size":0,"ondisk_log_size":8684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7407,"num_write_kb":29628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8672","reported_seq":8741,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.169448+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:29.169448+0000","last_peered":"2024-09-15T21:21:29.169448+0000","last_clean":"2024-09-15T21:21:29.169448+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:29.169448+0000","last_undegraded":"2024-09-15T21:21:29.169448+0000","last_fullsized":"2024-09-15T21:21:29.169448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8672,"log_dups_size":0,"ondisk_log_size":8672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":34688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8690","reported_seq":8806,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.546184+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:27.546184+0000","last_peered":"2024-09-15T21:21:27.546184+0000","last_clean":"2024-09-15T21:21:27.546184+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:27.546184+0000","last_undegraded":"2024-09-15T21:21:27.546184+0000","last_fullsized":"2024-09-15T21:21:27.546184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6548,"num_write_kb":26192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8540","reported_seq":8669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.547808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:27.547808+0000","last_peered":"2024-09-15T21:21:27.547808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:27.547808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8765","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.609999+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:27.609999+0000","last_peered":"2024-09-15T21:21:27.609999+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:27.609999+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8765,"log_dups_size":0,"ondisk_log_size":8765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8526","reported_seq":8645,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.568093+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:27.568093+0000","last_peered":"2024-09-15T21:21:27.568093+0000","last_clean":"2024-09-15T21:21:27.568093+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:27.568093+0000","last_undegraded":"2024-09-15T21:21:27.568093+0000","last_fullsized":"2024-09-15T21:21:27.568093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8526,"log_dups_size":0,"ondisk_log_size":8526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6311,"num_write_kb":25244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8552","reported_seq":8676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.116190+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:29.116190+0000","last_peered":"2024-09-15T21:21:29.116190+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:29.116190+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7138,"num_write_kb":28552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8377","reported_seq":8499,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.497991+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:27.497991+0000","last_peered":"2024-09-15T21:21:27.497991+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:27.497991+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279424,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8450","reported_seq":8559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.075638+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:29.075638+0000","last_peered":"2024-09-15T21:21:29.075638+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:29.075638+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7001,"num_write_kb":28004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8639","reported_seq":8752,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.130846+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:29.130846+0000","last_peered":"2024-09-15T21:21:29.130846+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:29.130846+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8639,"log_dups_size":0,"ondisk_log_size":8639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":28152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8646","reported_seq":8774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.588101+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:27.588101+0000","last_peered":"2024-09-15T21:21:27.588101+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:27.588101+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8646,"log_dups_size":0,"ondisk_log_size":8646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35405824,"num_objects":8644,"num_object_clones":0,"num_object_copies":17288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8554","reported_seq":8681,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.496725+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:27.496725+0000","last_peered":"2024-09-15T21:21:27.496725+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:27.496725+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8554,"log_dups_size":0,"ondisk_log_size":8554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8516","reported_seq":8638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.553800+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:27.553800+0000","last_peered":"2024-09-15T21:21:27.553800+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:27.553800+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8516,"log_dups_size":0,"ondisk_log_size":8516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8638","reported_seq":12741,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:27.598081+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:27.598081+0000","last_peered":"2024-09-15T21:21:27.598081+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:27.598081+0000","last_undegraded":"2024-09-15T21:21:27.598081+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332713,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1987,"num_bytes_recovered":8126464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7141},{"shards":"0,2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":562578025,"num_objects":137349,"num_object_clones":0,"num_object_copies":274698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108844,"num_write_kb":435376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5380,"num_bytes_recovered":22020096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1146855424,"data_stored":1146782365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137430,"ondisk_log_size":137430,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564656,"kb_used_data":384416,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93807184,"statfs":{"total":96636764160,"available":96058556416,"internally_reserved":0,"allocated":393641984,"data_stored":393376139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761228,"kb_used_data":464444,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93610612,"statfs":{"total":96636764160,"available":95857266688,"internally_reserved":0,"allocated":475590656,"data_stored":475324811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":303886001},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489888,"kb_used_data":269392,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93881952,"statfs":{"total":96636764160,"available":96135118848,"internally_reserved":0,"allocated":275857408,"data_stored":275528941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":476344320,"data_stored":476340841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276524491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393920512,"data_stored":393917033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:30.785 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:31.416 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:31.432 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-15T21:21:30.388435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":562578044,"num_objects":137351,"num_object_clones":0,"num_object_copies":274702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108846,"num_write_kb":435378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5381,"num_bytes_recovered":22020115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137433,"ondisk_log_size":137433,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815772,"kb_used_data":1118252,"kb_used_omap":12,"kb_used_meta":697459,"kb_avail":281299748,"statfs":{"total":289910292480,"available":288050941952,"internally_reserved":0,"allocated":1145090048,"data_stored":1144229891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":714198026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,4,6,8,11,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21757952,"num_objects":5312,"num_object_clones":0,"num_object_copies":10624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":5312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5312,"num_write_kb":21248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009559"},"pg_stats":[{"pgid":"2.f","version":"46'8527","reported_seq":8603,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.591347+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:27.591347+0000","last_peered":"2024-09-15T21:21:27.591347+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:27.591347+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34926592,"num_objects":8527,"num_object_clones":0,"num_object_copies":17054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8527,"num_write_kb":34108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8654","reported_seq":8750,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.116648+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:29.116648+0000","last_peered":"2024-09-15T21:21:29.116648+0000","last_clean":"2024-09-15T21:21:29.116648+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:29.116648+0000","last_undegraded":"2024-09-15T21:21:29.116648+0000","last_fullsized":"2024-09-15T21:21:29.116648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35422208,"num_objects":8648,"num_object_clones":0,"num_object_copies":17296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":29520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8684","reported_seq":15599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.547674+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:27.547674+0000","last_peered":"2024-09-15T21:21:27.547674+0000","last_clean":"2024-09-15T21:21:27.547674+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:27.547674+0000","last_undegraded":"2024-09-15T21:21:27.547674+0000","last_fullsized":"2024-09-15T21:21:27.547674+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8684,"log_dups_size":0,"ondisk_log_size":8684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7407,"num_write_kb":29628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8672","reported_seq":8741,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.169448+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:29.169448+0000","last_peered":"2024-09-15T21:21:29.169448+0000","last_clean":"2024-09-15T21:21:29.169448+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:29.169448+0000","last_undegraded":"2024-09-15T21:21:29.169448+0000","last_fullsized":"2024-09-15T21:21:29.169448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8672,"log_dups_size":0,"ondisk_log_size":8672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":34688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8690","reported_seq":8806,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.546184+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:27.546184+0000","last_peered":"2024-09-15T21:21:27.546184+0000","last_clean":"2024-09-15T21:21:27.546184+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:27.546184+0000","last_undegraded":"2024-09-15T21:21:27.546184+0000","last_fullsized":"2024-09-15T21:21:27.546184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6548,"num_write_kb":26192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8540","reported_seq":8669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.547808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:27.547808+0000","last_peered":"2024-09-15T21:21:27.547808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:27.547808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8765","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.609999+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:27.609999+0000","last_peered":"2024-09-15T21:21:27.609999+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:27.609999+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8765,"log_dups_size":0,"ondisk_log_size":8765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8526","reported_seq":8645,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.568093+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:27.568093+0000","last_peered":"2024-09-15T21:21:27.568093+0000","last_clean":"2024-09-15T21:21:27.568093+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:27.568093+0000","last_undegraded":"2024-09-15T21:21:27.568093+0000","last_fullsized":"2024-09-15T21:21:27.568093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8526,"log_dups_size":0,"ondisk_log_size":8526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6311,"num_write_kb":25244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8552","reported_seq":8676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.116190+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:29.116190+0000","last_peered":"2024-09-15T21:21:29.116190+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:29.116190+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7138,"num_write_kb":28552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8377","reported_seq":8499,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.497991+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:27.497991+0000","last_peered":"2024-09-15T21:21:27.497991+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:27.497991+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279424,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8450","reported_seq":8559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.075638+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:29.075638+0000","last_peered":"2024-09-15T21:21:29.075638+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:29.075638+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7001,"num_write_kb":28004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8639","reported_seq":8752,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.130846+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:29.130846+0000","last_peered":"2024-09-15T21:21:29.130846+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:29.130846+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8639,"log_dups_size":0,"ondisk_log_size":8639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":28152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8646","reported_seq":8774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.588101+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:27.588101+0000","last_peered":"2024-09-15T21:21:27.588101+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:27.588101+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8646,"log_dups_size":0,"ondisk_log_size":8646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35405824,"num_objects":8644,"num_object_clones":0,"num_object_copies":17288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8554","reported_seq":8681,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.496725+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:27.496725+0000","last_peered":"2024-09-15T21:21:27.496725+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:27.496725+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8554,"log_dups_size":0,"ondisk_log_size":8554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8516","reported_seq":8638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.553800+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:27.553800+0000","last_peered":"2024-09-15T21:21:27.553800+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:27.553800+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8516,"log_dups_size":0,"ondisk_log_size":8516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8638","reported_seq":12741,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:27.598081+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:27.598081+0000","last_peered":"2024-09-15T21:21:27.598081+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:27.598081+0000","last_undegraded":"2024-09-15T21:21:27.598081+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332713,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1987,"num_bytes_recovered":8126464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7141},{"shards":"0,2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":562578025,"num_objects":137349,"num_object_clones":0,"num_object_copies":274698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108844,"num_write_kb":435376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5380,"num_bytes_recovered":22020096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1146855424,"data_stored":1146782365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137430,"ondisk_log_size":137430,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564656,"kb_used_data":384416,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93807184,"statfs":{"total":96636764160,"available":96058556416,"internally_reserved":0,"allocated":393641984,"data_stored":393376139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761228,"kb_used_data":464444,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93610612,"statfs":{"total":96636764160,"available":95857266688,"internally_reserved":0,"allocated":475590656,"data_stored":475324811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":303886001},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489888,"kb_used_data":269392,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93881952,"statfs":{"total":96636764160,"available":96135118848,"internally_reserved":0,"allocated":275857408,"data_stored":275528941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":476344320,"data_stored":476340841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276524491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393920512,"data_stored":393917033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8527", 'reported_seq': 8603, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.591347+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:27.591347+0000', 'last_peered': '2024-09-15T21:21:27.591347+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:27.591347+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 8527, 'log_dups_size': 0, 'ondisk_log_size': 8527, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34926592, 'num_objects': 8527, 'num_object_clones': 0, 'num_object_copies': 17054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8527, 'num_write_kb': 34108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5160}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8540", 'reported_seq': 8669, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.547808+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:27.547808+0000', 'last_peered': '2024-09-15T21:21:27.547808+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:27.547808+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 8540, 'log_dups_size': 0, 'ondisk_log_size': 8540, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34959360, 'num_objects': 8535, 'num_object_clones': 0, 'num_object_copies': 17070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5965, 'num_write_kb': 23860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5164}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8765", 'reported_seq': 8888, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.609999+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:27.609999+0000', 'last_peered': '2024-09-15T21:21:27.609999+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:27.609999+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8765, 'log_dups_size': 0, 'ondisk_log_size': 8765, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35876864, 'num_objects': 8759, 'num_object_clones': 0, 'num_object_copies': 17518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6401, 'num_write_kb': 25604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5288}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8552", 'reported_seq': 8676, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:29.116190+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:29.116190+0000', 'last_peered': '2024-09-15T21:21:29.116190+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:29.116190+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8552, 'log_dups_size': 0, 'ondisk_log_size': 8552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35000320, 'num_objects': 8545, 'num_object_clones': 0, 'num_object_copies': 17090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7138, 'num_write_kb': 28552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5207}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8377", 'reported_seq': 8499, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.497991+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:27.497991+0000', 'last_peered': '2024-09-15T21:21:27.497991+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:27.497991+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8377, 'log_dups_size': 0, 'ondisk_log_size': 8377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34279424, 'num_objects': 8369, 'num_object_clones': 0, 'num_object_copies': 16738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5903, 'num_write_kb': 23612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5028}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8450", 'reported_seq': 8559, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:29.075638+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:29.075638+0000', 'last_peered': '2024-09-15T21:21:29.075638+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:29.075638+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8450, 'log_dups_size': 0, 'ondisk_log_size': 8450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34590720, 'num_objects': 8445, 'num_object_clones': 0, 'num_object_copies': 16890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7001, 'num_write_kb': 28004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 5179}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8639", 'reported_seq': 8752, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:29.130846+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:29.130846+0000', 'last_peered': '2024-09-15T21:21:29.130846+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:29.130846+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8639, 'log_dups_size': 0, 'ondisk_log_size': 8639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35360768, 'num_objects': 8633, 'num_object_clones': 0, 'num_object_copies': 17266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7038, 'num_write_kb': 28152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 5283}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:31.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8646", 'reported_seq': 8774, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.588101+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:27.588101+0000', 'last_peered': '2024-09-15T21:21:27.588101+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:27.588101+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8646, 'log_dups_size': 0, 'ondisk_log_size': 8646, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35405824, 'num_objects': 8644, 'num_object_clones': 0, 'num_object_copies': 17288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5953, 'num_write_kb': 23812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5211}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8554", 'reported_seq': 8681, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.496725+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:27.496725+0000', 'last_peered': '2024-09-15T21:21:27.496725+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:27.496725+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 8554, 'log_dups_size': 0, 'ondisk_log_size': 8554, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35004416, 'num_objects': 8546, 'num_object_clones': 0, 'num_object_copies': 17092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6139, 'num_write_kb': 24556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5167}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8516", 'reported_seq': 8638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:27.553800+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:27.553800+0000', 'last_peered': '2024-09-15T21:21:27.553800+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:27.553800+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 8516, 'log_dups_size': 0, 'ondisk_log_size': 8516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34865152, 'num_objects': 8512, 'num_object_clones': 0, 'num_object_copies': 17024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6163, 'num_write_kb': 24652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5147}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:31.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8638", 'reported_seq': 12741, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:27.598081+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:27.598081+0000', 'last_peered': '2024-09-15T21:21:27.598081+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:27.598081+0000', 'last_undegraded': '2024-09-15T21:21:27.598081+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8638, 'log_dups_size': 0, 'ondisk_log_size': 8638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35332713, 'num_objects': 8627, 'num_object_clones': 0, 'num_object_copies': 17254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1486, 'num_objects_unfound': 0, 'num_objects_dirty': 8627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6298, 'num_write_kb': 25192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1987, 'num_bytes_recovered': 8126464, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7141}, {'shards': '0,2', 'objects': 1486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:31.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:31.551 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 218 20 138668 138648 2.48342 1.1875 0.0167937 0.0314513 2024-09-15T21:21:32.332 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:32.332 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/274702 objects degraded (12.267%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":305,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137351,"data_bytes":562578044,"bytes_used":1859350528,"bytes_avail":288050941952,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":274702,"degraded_ratio":0.1226747530050746,"misplaced_objects":1486,"misplaced_total":274702,"misplaced_ratio":0.0054094982926953571,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109480,"recovering_keys_per_sec":0,"num_objects_recovered":324,"num_bytes_recovered":1314816,"num_keys_recovered":0,"write_bytes_sec":1811719,"read_op_per_sec":0,"write_op_per_sec":442},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:32.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:32.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:32.551 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 219 20 139114 139094 2.48004 1.74219 0.0381762 0.0314927 2024-09-15T21:21:33.202 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:33.202 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:33.294 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-15T21:21:32.388894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":562578044,"num_objects":137351,"num_object_clones":0,"num_object_copies":274702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108846,"num_write_kb":435378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5381,"num_bytes_recovered":22020115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137433,"ondisk_log_size":137433,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815772,"kb_used_data":1118252,"kb_used_omap":12,"kb_used_meta":697459,"kb_avail":281299748,"statfs":{"total":289910292480,"available":288050941952,"internally_reserved":0,"allocated":1145090048,"data_stored":1144229891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":714198026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,4,6,8,11,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18870272,"num_objects":4607,"num_object_clones":0,"num_object_copies":9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4607,"num_write_kb":18428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009532"},"pg_stats":[{"pgid":"2.f","version":"46'8527","reported_seq":8603,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.591347+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:27.591347+0000","last_peered":"2024-09-15T21:21:27.591347+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:27.591347+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34926592,"num_objects":8527,"num_object_clones":0,"num_object_copies":17054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8527,"num_write_kb":34108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5160},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8654","reported_seq":8750,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.116648+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:29.116648+0000","last_peered":"2024-09-15T21:21:29.116648+0000","last_clean":"2024-09-15T21:21:29.116648+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:29.116648+0000","last_undegraded":"2024-09-15T21:21:29.116648+0000","last_fullsized":"2024-09-15T21:21:29.116648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35422208,"num_objects":8648,"num_object_clones":0,"num_object_copies":17296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":29520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8684","reported_seq":15599,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.547674+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:27.547674+0000","last_peered":"2024-09-15T21:21:27.547674+0000","last_clean":"2024-09-15T21:21:27.547674+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:27.547674+0000","last_undegraded":"2024-09-15T21:21:27.547674+0000","last_fullsized":"2024-09-15T21:21:27.547674+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8684,"log_dups_size":0,"ondisk_log_size":8684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7407,"num_write_kb":29628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8672","reported_seq":8741,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:29.169448+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:29.169448+0000","last_peered":"2024-09-15T21:21:29.169448+0000","last_clean":"2024-09-15T21:21:29.169448+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:29.169448+0000","last_undegraded":"2024-09-15T21:21:29.169448+0000","last_fullsized":"2024-09-15T21:21:29.169448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8672,"log_dups_size":0,"ondisk_log_size":8672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":8672,"num_object_clones":0,"num_object_copies":17344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":34688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8690","reported_seq":8806,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.546184+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:27.546184+0000","last_peered":"2024-09-15T21:21:27.546184+0000","last_clean":"2024-09-15T21:21:27.546184+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:27.546184+0000","last_undegraded":"2024-09-15T21:21:27.546184+0000","last_fullsized":"2024-09-15T21:21:27.546184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6548,"num_write_kb":26192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8540","reported_seq":8669,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.547808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:27.547808+0000","last_peered":"2024-09-15T21:21:27.547808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:27.547808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8540,"log_dups_size":0,"ondisk_log_size":8540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8765","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.609999+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:27.609999+0000","last_peered":"2024-09-15T21:21:27.609999+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:27.609999+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8765,"log_dups_size":0,"ondisk_log_size":8765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5288},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8526","reported_seq":8645,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:27.568093+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:27.568093+0000","last_peered":"2024-09-15T21:21:27.568093+0000","last_clean":"2024-09-15T21:21:27.568093+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:27.568093+0000","last_undegraded":"2024-09-15T21:21:27.568093+0000","last_fullsized":"2024-09-15T21:21:27.568093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8526,"log_dups_size":0,"ondisk_log_size":8526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6311,"num_write_kb":25244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8552","reported_seq":8676,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.116190+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:29.116190+0000","last_peered":"2024-09-15T21:21:29.116190+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:29.116190+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000320,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7138,"num_write_kb":28552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8377","reported_seq":8499,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.497991+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:27.497991+0000","last_peered":"2024-09-15T21:21:27.497991+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:27.497991+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8377,"log_dups_size":0,"ondisk_log_size":8377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279424,"num_objects":8369,"num_object_clones":0,"num_object_copies":16738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5028},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":100,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461288+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461288+0000","last_peered":"2024-09-15T21:20:16.461288+0000","last_clean":"2024-09-15T21:20:16.461288+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461288+0000","last_undegraded":"2024-09-15T21:20:16.461288+0000","last_fullsized":"2024-09-15T21:20:16.461288+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8450","reported_seq":8559,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.075638+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:29.075638+0000","last_peered":"2024-09-15T21:21:29.075638+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:29.075638+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8450,"log_dups_size":0,"ondisk_log_size":8450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7001,"num_write_kb":28004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8639","reported_seq":8752,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.130846+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:29.130846+0000","last_peered":"2024-09-15T21:21:29.130846+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:29.130846+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8639,"log_dups_size":0,"ondisk_log_size":8639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7038,"num_write_kb":28152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8646","reported_seq":8774,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.588101+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:27.588101+0000","last_peered":"2024-09-15T21:21:27.588101+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:27.588101+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8646,"log_dups_size":0,"ondisk_log_size":8646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35405824,"num_objects":8644,"num_object_clones":0,"num_object_copies":17288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5953,"num_write_kb":23812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8554","reported_seq":8681,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.496725+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:27.496725+0000","last_peered":"2024-09-15T21:21:27.496725+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:27.496725+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8554,"log_dups_size":0,"ondisk_log_size":8554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35004416,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5167},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8516","reported_seq":8638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:27.553800+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:27.553800+0000","last_peered":"2024-09-15T21:21:27.553800+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:27.553800+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8516,"log_dups_size":0,"ondisk_log_size":8516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34865152,"num_objects":8512,"num_object_clones":0,"num_object_copies":17024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5147},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8638","reported_seq":12741,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:27.598081+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:27.598081+0000","last_peered":"2024-09-15T21:21:27.598081+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:27.598081+0000","last_undegraded":"2024-09-15T21:21:27.598081+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8638,"log_dups_size":0,"ondisk_log_size":8638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332713,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1987,"num_bytes_recovered":8126464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7141},{"shards":"0,2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":562578025,"num_objects":137349,"num_object_clones":0,"num_object_copies":274698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1486,"num_objects_unfound":0,"num_objects_dirty":137349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108844,"num_write_kb":435376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5380,"num_bytes_recovered":22020096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1146855424,"data_stored":1146782365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137430,"ondisk_log_size":137430,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953523,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564656,"kb_used_data":384416,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93807184,"statfs":{"total":96636764160,"available":96058556416,"internally_reserved":0,"allocated":393641984,"data_stored":393376139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149750,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761228,"kb_used_data":464444,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93610612,"statfs":{"total":96636764160,"available":95857266688,"internally_reserved":0,"allocated":475590656,"data_stored":475324811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":303886001},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489888,"kb_used_data":269392,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93881952,"statfs":{"total":96636764160,"available":96135118848,"internally_reserved":0,"allocated":275857408,"data_stored":275528941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":225767084},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":476344320,"data_stored":476340841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":276590592,"data_stored":276524491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393920512,"data_stored":393917033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:33.552 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:2024-09-15T21:21:33.551504+0000 min lat: 0.00128075 max lat: 4.27752 avg lat: 0.0315274 2024-09-15T21:21:33.552 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:21:33.552 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 220 20 139522 139502 2.47601 1.59375 0.008736 0.0315274 2024-09-15T21:21:34.554 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 221 20 139959 139939 2.47253 1.70703 0.0522293 0.0315867 2024-09-15T21:21:35.552 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 222 20 140455 140435 2.47012 1.9375 0.0675409 0.0316205 2024-09-15T21:21:36.296 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:36.556 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 223 20 140939 140919 2.46748 1.89062 0.0593021 0.0316539 2024-09-15T21:21:37.203 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:37.203 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:37.285 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-15T21:21:36.389828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570876540,"num_objects":139377,"num_object_clones":0,"num_object_copies":278754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":139377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110872,"num_write_kb":443482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22847507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139459,"ondisk_log_size":139459,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866272,"kb_used_data":1135404,"kb_used_omap":12,"kb_used_meta":730803,"kb_avail":281249248,"statfs":{"total":289910292480,"available":287999229952,"internally_reserved":0,"allocated":1162653696,"data_stored":1161793539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":748342282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,0,11,15,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-377,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4263,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009522"},"pg_stats":[{"pgid":"2.f","version":"46'8637","reported_seq":8713,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.620686+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:32.620686+0000","last_peered":"2024-09-15T21:21:32.620686+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:32.620686+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35377152,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8637,"num_write_kb":34548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5270},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8785","reported_seq":8881,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.189733+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:34.189733+0000","last_peered":"2024-09-15T21:21:34.189733+0000","last_clean":"2024-09-15T21:21:34.189733+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:34.189733+0000","last_undegraded":"2024-09-15T21:21:34.189733+0000","last_fullsized":"2024-09-15T21:21:34.189733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7511,"num_write_kb":30044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8819","reported_seq":15734,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.616624+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:32.616624+0000","last_peered":"2024-09-15T21:21:32.616624+0000","last_clean":"2024-09-15T21:21:32.616624+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:32.616624+0000","last_undegraded":"2024-09-15T21:21:32.616624+0000","last_fullsized":"2024-09-15T21:21:32.616624+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106240,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7542,"num_write_kb":30168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8786","reported_seq":8853,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.194415+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:34.194415+0000","last_peered":"2024-09-15T21:21:34.194415+0000","last_clean":"2024-09-15T21:21:34.194415+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:34.194415+0000","last_undegraded":"2024-09-15T21:21:34.194415+0000","last_fullsized":"2024-09-15T21:21:34.194415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8786,"log_dups_size":0,"ondisk_log_size":8786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8826","reported_seq":8944,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.681476+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:32.681476+0000","last_peered":"2024-09-15T21:21:32.681476+0000","last_clean":"2024-09-15T21:21:32.681476+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:32.681476+0000","last_undegraded":"2024-09-15T21:21:32.681476+0000","last_fullsized":"2024-09-15T21:21:32.681476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8826,"log_dups_size":0,"ondisk_log_size":8826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36122624,"num_objects":8819,"num_object_clones":0,"num_object_copies":17638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6684,"num_write_kb":26736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8654","reported_seq":8783,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.592875+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:32.592875+0000","last_peered":"2024-09-15T21:21:32.592875+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:32.592875+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6079,"num_write_kb":24316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5278},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8897","reported_seq":9018,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.670272+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:32.670272+0000","last_peered":"2024-09-15T21:21:32.670272+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:32.670272+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8897,"log_dups_size":0,"ondisk_log_size":8897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36417536,"num_objects":8891,"num_object_clones":0,"num_object_copies":17782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8657","reported_seq":8776,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.481023+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:32.481023+0000","last_peered":"2024-09-15T21:21:32.481023+0000","last_clean":"2024-09-15T21:21:32.481023+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:32.481023+0000","last_undegraded":"2024-09-15T21:21:32.481023+0000","last_fullsized":"2024-09-15T21:21:32.481023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8664","reported_seq":8786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.199092+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:34.199092+0000","last_peered":"2024-09-15T21:21:34.199092+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:34.199092+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8664,"log_dups_size":0,"ondisk_log_size":8664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459072,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":29000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8508","reported_seq":8629,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.697291+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:32.697291+0000","last_peered":"2024-09-15T21:21:32.697291+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:32.697291+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8508,"log_dups_size":0,"ondisk_log_size":8508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6034,"num_write_kb":24136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8576","reported_seq":8685,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.167530+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:34.167530+0000","last_peered":"2024-09-15T21:21:34.167530+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:34.167530+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7127,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8774","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.135400+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:34.135400+0000","last_peered":"2024-09-15T21:21:34.135400+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:34.135400+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5418}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8772","reported_seq":8900,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.652235+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:32.652235+0000","last_peered":"2024-09-15T21:21:32.652235+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:32.652235+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6079,"num_write_kb":24316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8687","reported_seq":8816,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.688478+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:32.688478+0000","last_peered":"2024-09-15T21:21:32.688478+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:32.688478+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8687,"log_dups_size":0,"ondisk_log_size":8687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35549184,"num_objects":8679,"num_object_clones":0,"num_object_copies":17358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6272,"num_write_kb":25088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5300},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8662","reported_seq":8785,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.683579+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:32.683579+0000","last_peered":"2024-09-15T21:21:32.683579+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:32.683579+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6309,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5293},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8752","reported_seq":13259,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:32.720419+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:32.720419+0000","last_peered":"2024-09-15T21:21:32.720419+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:32.720419+0000","last_undegraded":"2024-09-15T21:21:32.720419+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8752,"log_dups_size":0,"ondisk_log_size":8752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35799657,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6412,"num_write_kb":25648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2189,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7457},{"shards":"0,2","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570876521,"num_objects":139375,"num_object_clones":0,"num_object_copies":278750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":139375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110870,"num_write_kb":443480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22847488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164292096,"data_stored":1164219037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139456,"ondisk_log_size":139456,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627004,"kb_used_data":388460,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93744836,"statfs":{"total":96636764160,"available":95994712064,"internally_reserved":0,"allocated":397783040,"data_stored":397517195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754904,"kb_used_data":470600,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93616936,"statfs":{"total":96636764160,"available":95863742464,"internally_reserved":0,"allocated":481894400,"data_stored":481628555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":291106481},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484364,"kb_used_data":276344,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93887476,"statfs":{"total":96636764160,"available":96140775424,"internally_reserved":0,"allocated":282976256,"data_stored":282647789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":212987564},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":482607104,"data_stored":482603625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283713536,"data_stored":283647435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397971456,"data_stored":397967977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:37.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:37.560 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 224 20 141394 141374 2.46435 1.77734 0.0444236 0.0316951 2024-09-15T21:21:38.237 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:38.237 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:38.291 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-15T21:21:36.389828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570876540,"num_objects":139377,"num_object_clones":0,"num_object_copies":278754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":139377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110872,"num_write_kb":443482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5583,"num_bytes_recovered":22847507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139459,"ondisk_log_size":139459,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1866272,"kb_used_data":1135404,"kb_used_omap":12,"kb_used_meta":730803,"kb_avail":281249248,"statfs":{"total":289910292480,"available":287999229952,"internally_reserved":0,"allocated":1162653696,"data_stored":1161793539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":748342282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,0,11,15,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-377,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4263,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009522"},"pg_stats":[{"pgid":"2.f","version":"46'8637","reported_seq":8713,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.620686+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:32.620686+0000","last_peered":"2024-09-15T21:21:32.620686+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:32.620686+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35377152,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8637,"num_write_kb":34548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5270},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8785","reported_seq":8881,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.189733+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:34.189733+0000","last_peered":"2024-09-15T21:21:34.189733+0000","last_clean":"2024-09-15T21:21:34.189733+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:34.189733+0000","last_undegraded":"2024-09-15T21:21:34.189733+0000","last_fullsized":"2024-09-15T21:21:34.189733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7511,"num_write_kb":30044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8819","reported_seq":15734,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.616624+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:32.616624+0000","last_peered":"2024-09-15T21:21:32.616624+0000","last_clean":"2024-09-15T21:21:32.616624+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:32.616624+0000","last_undegraded":"2024-09-15T21:21:32.616624+0000","last_fullsized":"2024-09-15T21:21:32.616624+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8819,"log_dups_size":0,"ondisk_log_size":8819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106240,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7542,"num_write_kb":30168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8786","reported_seq":8853,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.194415+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:34.194415+0000","last_peered":"2024-09-15T21:21:34.194415+0000","last_clean":"2024-09-15T21:21:34.194415+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:34.194415+0000","last_undegraded":"2024-09-15T21:21:34.194415+0000","last_fullsized":"2024-09-15T21:21:34.194415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8786,"log_dups_size":0,"ondisk_log_size":8786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8826","reported_seq":8944,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.681476+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:32.681476+0000","last_peered":"2024-09-15T21:21:32.681476+0000","last_clean":"2024-09-15T21:21:32.681476+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:32.681476+0000","last_undegraded":"2024-09-15T21:21:32.681476+0000","last_fullsized":"2024-09-15T21:21:32.681476+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8826,"log_dups_size":0,"ondisk_log_size":8826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36122624,"num_objects":8819,"num_object_clones":0,"num_object_copies":17638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6684,"num_write_kb":26736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8654","reported_seq":8783,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.592875+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:32.592875+0000","last_peered":"2024-09-15T21:21:32.592875+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:32.592875+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8654,"log_dups_size":0,"ondisk_log_size":8654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6079,"num_write_kb":24316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5278},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'8897","reported_seq":9018,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.670272+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:32.670272+0000","last_peered":"2024-09-15T21:21:32.670272+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:32.670272+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":8897,"log_dups_size":0,"ondisk_log_size":8897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36417536,"num_objects":8891,"num_object_clones":0,"num_object_copies":17782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8657","reported_seq":8776,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:32.481023+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:32.481023+0000","last_peered":"2024-09-15T21:21:32.481023+0000","last_clean":"2024-09-15T21:21:32.481023+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:32.481023+0000","last_undegraded":"2024-09-15T21:21:32.481023+0000","last_fullsized":"2024-09-15T21:21:32.481023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8664","reported_seq":8786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.199092+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:34.199092+0000","last_peered":"2024-09-15T21:21:34.199092+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:34.199092+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8664,"log_dups_size":0,"ondisk_log_size":8664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459072,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":29000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8508","reported_seq":8629,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.697291+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:32.697291+0000","last_peered":"2024-09-15T21:21:32.697291+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:32.697291+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8508,"log_dups_size":0,"ondisk_log_size":8508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6034,"num_write_kb":24136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8576","reported_seq":8685,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.167530+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:34.167530+0000","last_peered":"2024-09-15T21:21:34.167530+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:34.167530+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7127,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8774","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.135400+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:34.135400+0000","last_peered":"2024-09-15T21:21:34.135400+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:34.135400+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5418}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8772","reported_seq":8900,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.652235+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:32.652235+0000","last_peered":"2024-09-15T21:21:32.652235+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:32.652235+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8772,"log_dups_size":0,"ondisk_log_size":8772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35921920,"num_objects":8770,"num_object_clones":0,"num_object_copies":17540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6079,"num_write_kb":24316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8687","reported_seq":8816,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.688478+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:32.688478+0000","last_peered":"2024-09-15T21:21:32.688478+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:32.688478+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8687,"log_dups_size":0,"ondisk_log_size":8687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35549184,"num_objects":8679,"num_object_clones":0,"num_object_copies":17358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6272,"num_write_kb":25088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5300},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8662","reported_seq":8785,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:32.683579+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:32.683579+0000","last_peered":"2024-09-15T21:21:32.683579+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:32.683579+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8662,"log_dups_size":0,"ondisk_log_size":8662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6309,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5293},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8752","reported_seq":13259,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:32.720419+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:32.720419+0000","last_peered":"2024-09-15T21:21:32.720419+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:32.720419+0000","last_undegraded":"2024-09-15T21:21:32.720419+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8752,"log_dups_size":0,"ondisk_log_size":8752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35799657,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6412,"num_write_kb":25648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2189,"num_bytes_recovered":8953856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7457},{"shards":"0,2","objects":1284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570876521,"num_objects":139375,"num_object_clones":0,"num_object_copies":278750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1284,"num_objects_unfound":0,"num_objects_dirty":139375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110870,"num_write_kb":443480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22847488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164292096,"data_stored":1164219037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139456,"ondisk_log_size":139456,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627004,"kb_used_data":388460,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93744836,"statfs":{"total":96636764160,"available":95994712064,"internally_reserved":0,"allocated":397783040,"data_stored":397517195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149751,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754904,"kb_used_data":470600,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93616936,"statfs":{"total":96636764160,"available":95863742464,"internally_reserved":0,"allocated":481894400,"data_stored":481628555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":291106481},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484364,"kb_used_data":276344,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93887476,"statfs":{"total":96636764160,"available":96140775424,"internally_reserved":0,"allocated":282976256,"data_stored":282647789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":212987564},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":482607104,"data_stored":482603625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":283713536,"data_stored":283647435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397971456,"data_stored":397967977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:38.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:38.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8637", 'reported_seq': 8713, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.620686+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:32.620686+0000', 'last_peered': '2024-09-15T21:21:32.620686+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:32.620686+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 8637, 'log_dups_size': 0, 'ondisk_log_size': 8637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35377152, 'num_objects': 8637, 'num_object_clones': 0, 'num_object_copies': 17274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8637, 'num_write_kb': 34548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5270}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8654", 'reported_seq': 8783, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.592875+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:32.592875+0000', 'last_peered': '2024-09-15T21:21:32.592875+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:32.592875+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 8654, 'log_dups_size': 0, 'ondisk_log_size': 8654, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35426304, 'num_objects': 8649, 'num_object_clones': 0, 'num_object_copies': 17298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6079, 'num_write_kb': 24316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5278}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'8897", 'reported_seq': 9018, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.670272+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:32.670272+0000', 'last_peered': '2024-09-15T21:21:32.670272+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:32.670272+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 8897, 'log_dups_size': 0, 'ondisk_log_size': 8897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36417536, 'num_objects': 8891, 'num_object_clones': 0, 'num_object_copies': 17782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6533, 'num_write_kb': 26132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5420}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8664", 'reported_seq': 8786, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:34.199092+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:34.199092+0000', 'last_peered': '2024-09-15T21:21:34.199092+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:34.199092+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8664, 'log_dups_size': 0, 'ondisk_log_size': 8664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35459072, 'num_objects': 8657, 'num_object_clones': 0, 'num_object_copies': 17314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7250, 'num_write_kb': 29000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:38.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:38.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8508", 'reported_seq': 8629, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.697291+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:32.697291+0000', 'last_peered': '2024-09-15T21:21:32.697291+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:32.697291+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8508, 'log_dups_size': 0, 'ondisk_log_size': 8508, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34816000, 'num_objects': 8500, 'num_object_clones': 0, 'num_object_copies': 17000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6034, 'num_write_kb': 24136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5159}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:38.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8576", 'reported_seq': 8685, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:34.167530+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:34.167530+0000', 'last_peered': '2024-09-15T21:21:34.167530+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:34.167530+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8576, 'log_dups_size': 0, 'ondisk_log_size': 8576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35106816, 'num_objects': 8571, 'num_object_clones': 0, 'num_object_copies': 17142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7127, 'num_write_kb': 28508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 5305}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:38.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:38.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8774", 'reported_seq': 8888, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:34.135400+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:34.135400+0000', 'last_peered': '2024-09-15T21:21:34.135400+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:34.135400+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8774, 'log_dups_size': 0, 'ondisk_log_size': 8774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35913728, 'num_objects': 8768, 'num_object_clones': 0, 'num_object_copies': 17536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7173, 'num_write_kb': 28692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 5418}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:38.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:38.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8772", 'reported_seq': 8900, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.652235+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:32.652235+0000', 'last_peered': '2024-09-15T21:21:32.652235+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:32.652235+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8772, 'log_dups_size': 0, 'ondisk_log_size': 8772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35921920, 'num_objects': 8770, 'num_object_clones': 0, 'num_object_copies': 17540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6079, 'num_write_kb': 24316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5337}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:38.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8687", 'reported_seq': 8816, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.688478+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:32.688478+0000', 'last_peered': '2024-09-15T21:21:32.688478+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:32.688478+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 8687, 'log_dups_size': 0, 'ondisk_log_size': 8687, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35549184, 'num_objects': 8679, 'num_object_clones': 0, 'num_object_copies': 17358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6272, 'num_write_kb': 25088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5300}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:38.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8662", 'reported_seq': 8785, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:32.683579+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:32.683579+0000', 'last_peered': '2024-09-15T21:21:32.683579+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:32.683579+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 8662, 'log_dups_size': 0, 'ondisk_log_size': 8662, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35463168, 'num_objects': 8658, 'num_object_clones': 0, 'num_object_copies': 17316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6309, 'num_write_kb': 25236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5293}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:38.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8752", 'reported_seq': 13259, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:32.720419+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:32.720419+0000', 'last_peered': '2024-09-15T21:21:32.720419+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:32.720419+0000', 'last_undegraded': '2024-09-15T21:21:32.720419+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8752, 'log_dups_size': 0, 'ondisk_log_size': 8752, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35799657, 'num_objects': 8741, 'num_object_clones': 0, 'num_object_copies': 17482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1284, 'num_objects_unfound': 0, 'num_objects_dirty': 8741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6412, 'num_write_kb': 25648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2189, 'num_bytes_recovered': 8953856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7457}, {'shards': '0,2', 'objects': 1284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:38.298 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:38.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 225 20 141858 141838 2.46145 1.8125 0.0661169 0.0317309 2024-09-15T21:21:39.358 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:39.359 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/281896 objects degraded (11.954%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":312,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140948,"data_bytes":577311356,"bytes_used":1972359168,"bytes_avail":287937933312,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":281896,"degraded_ratio":0.11954408718108805,"misplaced_objects":1121,"misplaced_total":281896,"misplaced_ratio":0.0039766438686607824,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":184175,"recovering_keys_per_sec":0,"num_objects_recovered":540,"num_bytes_recovered":2211840,"num_keys_recovered":0,"write_bytes_sec":1989772,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:39.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:39.427 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:39.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 226 19 142318 142299 2.45853 1.80078 0.0614106 0.0317699 2024-09-15T21:21:40.277 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:40.277 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:40.325 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-15T21:21:38.390267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577311356,"num_objects":140948,"num_object_clones":0,"num_object_copies":281896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":140948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112443,"num_write_kb":449766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5746,"num_bytes_recovered":23515155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141030,"ondisk_log_size":141030,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926132,"kb_used_data":1150852,"kb_used_omap":12,"kb_used_meta":775219,"kb_avail":281189388,"statfs":{"total":289910292480,"available":287937933312,"internally_reserved":0,"allocated":1178472448,"data_stored":1177612291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":793824266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,4,3,13,6,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-540,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5834,"num_write_kb":23336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":2211840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009447"},"pg_stats":[{"pgid":"2.f","version":"46'8780","reported_seq":8856,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.584190+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:37.584190+0000","last_peered":"2024-09-15T21:21:37.584190+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:37.584190+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8780,"log_dups_size":0,"ondisk_log_size":8780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8780,"num_write_kb":35120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8785","reported_seq":8881,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.189733+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:34.189733+0000","last_peered":"2024-09-15T21:21:34.189733+0000","last_clean":"2024-09-15T21:21:34.189733+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:34.189733+0000","last_undegraded":"2024-09-15T21:21:34.189733+0000","last_fullsized":"2024-09-15T21:21:34.189733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35958784,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7511,"num_write_kb":30044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8961","reported_seq":15876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.661480+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:37.661480+0000","last_peered":"2024-09-15T21:21:37.661480+0000","last_clean":"2024-09-15T21:21:37.661480+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:37.661480+0000","last_undegraded":"2024-09-15T21:21:37.661480+0000","last_fullsized":"2024-09-15T21:21:37.661480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8961,"log_dups_size":0,"ondisk_log_size":8961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36687872,"num_objects":8957,"num_object_clones":0,"num_object_copies":17914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":30736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8786","reported_seq":8853,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:34.194415+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:34.194415+0000","last_peered":"2024-09-15T21:21:34.194415+0000","last_clean":"2024-09-15T21:21:34.194415+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:34.194415+0000","last_undegraded":"2024-09-15T21:21:34.194415+0000","last_fullsized":"2024-09-15T21:21:34.194415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8786,"log_dups_size":0,"ondisk_log_size":8786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35987456,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8786,"num_write_kb":35144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8958","reported_seq":9076,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.672393+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:37.672393+0000","last_peered":"2024-09-15T21:21:37.672393+0000","last_clean":"2024-09-15T21:21:37.672393+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:37.672393+0000","last_undegraded":"2024-09-15T21:21:37.672393+0000","last_fullsized":"2024-09-15T21:21:37.672393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":27264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8785","reported_seq":8914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.671033+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:37.671033+0000","last_peered":"2024-09-15T21:21:37.671033+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:37.671033+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6210,"num_write_kb":24840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5409},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9040","reported_seq":9163,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.681793+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:37.681793+0000","last_peered":"2024-09-15T21:21:37.681793+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:37.681793+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6676,"num_write_kb":26704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8806","reported_seq":8922,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.697250+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:37.697250+0000","last_peered":"2024-09-15T21:21:37.697250+0000","last_clean":"2024-09-15T21:21:37.697250+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:37.697250+0000","last_undegraded":"2024-09-15T21:21:37.697250+0000","last_fullsized":"2024-09-15T21:21:37.697250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061184,"num_objects":8804,"num_object_clones":0,"num_object_copies":17608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8664","reported_seq":8786,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.199092+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:34.199092+0000","last_peered":"2024-09-15T21:21:34.199092+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:34.199092+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8664,"log_dups_size":0,"ondisk_log_size":8664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35459072,"num_objects":8657,"num_object_clones":0,"num_object_copies":17314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":29000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8650","reported_seq":8773,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.700878+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:37.700878+0000","last_peered":"2024-09-15T21:21:37.700878+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:37.700878+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8650,"log_dups_size":0,"ondisk_log_size":8650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5301},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8576","reported_seq":8685,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.167530+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:34.167530+0000","last_peered":"2024-09-15T21:21:34.167530+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:34.167530+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35106816,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7127,"num_write_kb":28508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8774","reported_seq":8888,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:34.135400+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:34.135400+0000","last_peered":"2024-09-15T21:21:34.135400+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:34.135400+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8774,"log_dups_size":0,"ondisk_log_size":8774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5418}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8925","reported_seq":9051,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.709180+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:37.709180+0000","last_peered":"2024-09-15T21:21:37.709180+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:37.709180+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6232,"num_write_kb":24928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5490},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8834","reported_seq":8962,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.743976+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:37.743976+0000","last_peered":"2024-09-15T21:21:37.743976+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:37.743976+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8797","reported_seq":8920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.664170+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:37.664170+0000","last_peered":"2024-09-15T21:21:37.664170+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:37.664170+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8797,"log_dups_size":0,"ondisk_log_size":8797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36016128,"num_objects":8793,"num_object_clones":0,"num_object_copies":17586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6444,"num_write_kb":25776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8906","reported_seq":13739,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:37.748090+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:37.748090+0000","last_peered":"2024-09-15T21:21:37.748090+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:37.748090+0000","last_undegraded":"2024-09-15T21:21:37.748090+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36430441,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":9621504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577311337,"num_objects":140946,"num_object_clones":0,"num_object_copies":281892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":140946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112441,"num_write_kb":449764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5745,"num_bytes_recovered":23515136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1179095040,"data_stored":1179021981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141027,"ondisk_log_size":141027,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627004,"kb_used_data":388460,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93744836,"statfs":{"total":96636764160,"available":95994712064,"internally_reserved":0,"allocated":397783040,"data_stored":397517195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785080,"kb_used_data":478568,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93586760,"statfs":{"total":96636764160,"available":95832842240,"internally_reserved":0,"allocated":490053632,"data_stored":489787787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514048,"kb_used_data":283824,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93857792,"statfs":{"total":96636764160,"available":96110379008,"internally_reserved":0,"allocated":290635776,"data_stored":290307309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":235728556},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489721856,"data_stored":489718377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291401728,"data_stored":291335627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397971456,"data_stored":397967977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:40.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 227 20 142774 142754 2.45553 1.77734 0.0199331 0.0318074 2024-09-15T21:21:41.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 228 20 143256 143236 2.45302 1.88281 0.0778031 0.0318417 2024-09-15T21:21:42.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 229 20 143734 143714 2.45046 1.86719 0.0448955 0.031873 2024-09-15T21:21:43.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:43.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 230 20 144121 144101 2.44638 1.51172 0.0575254 0.0319276 2024-09-15T21:21:44.017 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:44.017 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:44.036 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-15T21:21:42.393754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580321916,"num_objects":141683,"num_object_clones":0,"num_object_copies":283366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":141683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113178,"num_write_kb":452706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5746,"num_bytes_recovered":23515155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141765,"ondisk_log_size":141765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1934972,"kb_used_data":1155532,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281180548,"statfs":{"total":289910292480,"available":287928881152,"internally_reserved":0,"allocated":1183264768,"data_stored":1182404611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":798084106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,2,15,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17743872,"num_objects":4332,"num_object_clones":0,"num_object_copies":8664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-365,"num_objects_unfound":0,"num_objects_dirty":4332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4332,"num_write_kb":17328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005318"},"pg_stats":[{"pgid":"2.f","version":"46'8780","reported_seq":8856,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.584190+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:37.584190+0000","last_peered":"2024-09-15T21:21:37.584190+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:37.584190+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8780,"log_dups_size":0,"ondisk_log_size":8780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8780,"num_write_kb":35120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8934","reported_seq":9030,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:39.147530+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:39.147530+0000","last_peered":"2024-09-15T21:21:39.147530+0000","last_clean":"2024-09-15T21:21:39.147530+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:39.147530+0000","last_undegraded":"2024-09-15T21:21:39.147530+0000","last_fullsized":"2024-09-15T21:21:39.147530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":8928,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8961","reported_seq":15876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.661480+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:37.661480+0000","last_peered":"2024-09-15T21:21:37.661480+0000","last_clean":"2024-09-15T21:21:37.661480+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:37.661480+0000","last_undegraded":"2024-09-15T21:21:37.661480+0000","last_fullsized":"2024-09-15T21:21:37.661480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8961,"log_dups_size":0,"ondisk_log_size":8961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36687872,"num_objects":8957,"num_object_clones":0,"num_object_copies":17914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":30736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8938","reported_seq":9006,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:39.193254+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:39.193254+0000","last_peered":"2024-09-15T21:21:39.193254+0000","last_clean":"2024-09-15T21:21:39.193254+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:39.193254+0000","last_undegraded":"2024-09-15T21:21:39.193254+0000","last_fullsized":"2024-09-15T21:21:39.193254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8938,"log_dups_size":0,"ondisk_log_size":8938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36610048,"num_objects":8938,"num_object_clones":0,"num_object_copies":17876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8938,"num_write_kb":35752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8958","reported_seq":9076,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.672393+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:37.672393+0000","last_peered":"2024-09-15T21:21:37.672393+0000","last_clean":"2024-09-15T21:21:37.672393+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:37.672393+0000","last_undegraded":"2024-09-15T21:21:37.672393+0000","last_fullsized":"2024-09-15T21:21:37.672393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":27264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8785","reported_seq":8914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.671033+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:37.671033+0000","last_peered":"2024-09-15T21:21:37.671033+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:37.671033+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6210,"num_write_kb":24840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5409},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9040","reported_seq":9163,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.681793+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:37.681793+0000","last_peered":"2024-09-15T21:21:37.681793+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:37.681793+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6676,"num_write_kb":26704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8806","reported_seq":8922,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.697250+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:37.697250+0000","last_peered":"2024-09-15T21:21:37.697250+0000","last_clean":"2024-09-15T21:21:37.697250+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:37.697250+0000","last_undegraded":"2024-09-15T21:21:37.697250+0000","last_fullsized":"2024-09-15T21:21:37.697250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061184,"num_objects":8804,"num_object_clones":0,"num_object_copies":17608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8805","reported_seq":8929,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.212969+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:39.212969+0000","last_peered":"2024-09-15T21:21:39.212969+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:39.212969+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8805,"log_dups_size":0,"ondisk_log_size":8805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5460}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8650","reported_seq":8773,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.700878+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:37.700878+0000","last_peered":"2024-09-15T21:21:37.700878+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:37.700878+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8650,"log_dups_size":0,"ondisk_log_size":8650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5301},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8746","reported_seq":8855,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.081677+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:39.081677+0000","last_peered":"2024-09-15T21:21:39.081677+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:39.081677+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8746,"log_dups_size":0,"ondisk_log_size":8746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803136,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5475}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8897","reported_seq":9012,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.191262+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:39.191262+0000","last_peered":"2024-09-15T21:21:39.191262+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:39.191262+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8897,"log_dups_size":0,"ondisk_log_size":8897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36417536,"num_objects":8891,"num_object_clones":0,"num_object_copies":17782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5541}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8925","reported_seq":9051,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.709180+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:37.709180+0000","last_peered":"2024-09-15T21:21:37.709180+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:37.709180+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6232,"num_write_kb":24928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5490},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8834","reported_seq":8962,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.743976+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:37.743976+0000","last_peered":"2024-09-15T21:21:37.743976+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:37.743976+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8797","reported_seq":8920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.664170+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:37.664170+0000","last_peered":"2024-09-15T21:21:37.664170+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:37.664170+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8797,"log_dups_size":0,"ondisk_log_size":8797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36016128,"num_objects":8793,"num_object_clones":0,"num_object_copies":17586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6444,"num_write_kb":25776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8906","reported_seq":13739,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:37.748090+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:37.748090+0000","last_peered":"2024-09-15T21:21:37.748090+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:37.748090+0000","last_undegraded":"2024-09-15T21:21:37.748090+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36430441,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":9621504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580321897,"num_objects":141681,"num_object_clones":0,"num_object_copies":283362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":141681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113176,"num_write_kb":452704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5745,"num_bytes_recovered":23515136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1183952896,"data_stored":1183879837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141762,"ondisk_log_size":141762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635844,"kb_used_data":393140,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93735996,"statfs":{"total":96636764160,"available":95985659904,"internally_reserved":0,"allocated":402575360,"data_stored":402309515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":248508077},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785080,"kb_used_data":478568,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93586760,"statfs":{"total":96636764160,"available":95832842240,"internally_reserved":0,"allocated":490053632,"data_stored":489787787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514048,"kb_used_data":283824,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93857792,"statfs":{"total":96636764160,"available":96110379008,"internally_reserved":0,"allocated":290635776,"data_stored":290307309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":235728556},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489721856,"data_stored":489718377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291401728,"data_stored":291335627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402829312,"data_stored":402825833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:44.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:44.339 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:44.339 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:44.356 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-15T21:21:42.393754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580321916,"num_objects":141683,"num_object_clones":0,"num_object_copies":283366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":141683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113178,"num_write_kb":452706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5746,"num_bytes_recovered":23515155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141765,"ondisk_log_size":141765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1934972,"kb_used_data":1155532,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281180548,"statfs":{"total":289910292480,"available":287928881152,"internally_reserved":0,"allocated":1183264768,"data_stored":1182404611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":798084106},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,2,15,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17743872,"num_objects":4332,"num_object_clones":0,"num_object_copies":8664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-365,"num_objects_unfound":0,"num_objects_dirty":4332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4332,"num_write_kb":17328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005318"},"pg_stats":[{"pgid":"2.f","version":"46'8780","reported_seq":8856,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.584190+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:37.584190+0000","last_peered":"2024-09-15T21:21:37.584190+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:37.584190+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8780,"log_dups_size":0,"ondisk_log_size":8780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8780,"num_write_kb":35120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'8934","reported_seq":9030,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:39.147530+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:39.147530+0000","last_peered":"2024-09-15T21:21:39.147530+0000","last_clean":"2024-09-15T21:21:39.147530+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:39.147530+0000","last_undegraded":"2024-09-15T21:21:39.147530+0000","last_fullsized":"2024-09-15T21:21:39.147530+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36569088,"num_objects":8928,"num_object_clones":0,"num_object_copies":17856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'8961","reported_seq":15876,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.661480+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:37.661480+0000","last_peered":"2024-09-15T21:21:37.661480+0000","last_clean":"2024-09-15T21:21:37.661480+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:37.661480+0000","last_undegraded":"2024-09-15T21:21:37.661480+0000","last_fullsized":"2024-09-15T21:21:37.661480+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":8961,"log_dups_size":0,"ondisk_log_size":8961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36687872,"num_objects":8957,"num_object_clones":0,"num_object_copies":17914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":30736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'8938","reported_seq":9006,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:39.193254+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:39.193254+0000","last_peered":"2024-09-15T21:21:39.193254+0000","last_clean":"2024-09-15T21:21:39.193254+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:39.193254+0000","last_undegraded":"2024-09-15T21:21:39.193254+0000","last_fullsized":"2024-09-15T21:21:39.193254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8938,"log_dups_size":0,"ondisk_log_size":8938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36610048,"num_objects":8938,"num_object_clones":0,"num_object_copies":17876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8938,"num_write_kb":35752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'8958","reported_seq":9076,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.672393+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:37.672393+0000","last_peered":"2024-09-15T21:21:37.672393+0000","last_clean":"2024-09-15T21:21:37.672393+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:37.672393+0000","last_undegraded":"2024-09-15T21:21:37.672393+0000","last_fullsized":"2024-09-15T21:21:37.672393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":8958,"log_dups_size":0,"ondisk_log_size":8958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6816,"num_write_kb":27264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8785","reported_seq":8914,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.671033+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:37.671033+0000","last_peered":"2024-09-15T21:21:37.671033+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:37.671033+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6210,"num_write_kb":24840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5409},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9040","reported_seq":9163,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.681793+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:37.681793+0000","last_peered":"2024-09-15T21:21:37.681793+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:37.681793+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6676,"num_write_kb":26704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8806","reported_seq":8922,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:37.697250+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:37.697250+0000","last_peered":"2024-09-15T21:21:37.697250+0000","last_clean":"2024-09-15T21:21:37.697250+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:37.697250+0000","last_undegraded":"2024-09-15T21:21:37.697250+0000","last_fullsized":"2024-09-15T21:21:37.697250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8806,"log_dups_size":0,"ondisk_log_size":8806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36061184,"num_objects":8804,"num_object_clones":0,"num_object_copies":17608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8805","reported_seq":8929,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.212969+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:39.212969+0000","last_peered":"2024-09-15T21:21:39.212969+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:39.212969+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8805,"log_dups_size":0,"ondisk_log_size":8805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5460}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8650","reported_seq":8773,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.700878+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:37.700878+0000","last_peered":"2024-09-15T21:21:37.700878+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:37.700878+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8650,"log_dups_size":0,"ondisk_log_size":8650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6176,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5301},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8746","reported_seq":8855,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.081677+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:39.081677+0000","last_peered":"2024-09-15T21:21:39.081677+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:39.081677+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8746,"log_dups_size":0,"ondisk_log_size":8746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803136,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5475}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'8897","reported_seq":9012,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:39.191262+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:39.191262+0000","last_peered":"2024-09-15T21:21:39.191262+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:39.191262+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":8897,"log_dups_size":0,"ondisk_log_size":8897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36417536,"num_objects":8891,"num_object_clones":0,"num_object_copies":17782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5541}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'8925","reported_seq":9051,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.709180+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:37.709180+0000","last_peered":"2024-09-15T21:21:37.709180+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:37.709180+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6232,"num_write_kb":24928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5490},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8834","reported_seq":8962,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.743976+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:37.743976+0000","last_peered":"2024-09-15T21:21:37.743976+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:37.743976+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6419,"num_write_kb":25676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8797","reported_seq":8920,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:37.664170+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:37.664170+0000","last_peered":"2024-09-15T21:21:37.664170+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:37.664170+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8797,"log_dups_size":0,"ondisk_log_size":8797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36016128,"num_objects":8793,"num_object_clones":0,"num_object_copies":17586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6444,"num_write_kb":25776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'8906","reported_seq":13739,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:37.748090+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:37.748090+0000","last_peered":"2024-09-15T21:21:37.748090+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:37.748090+0000","last_undegraded":"2024-09-15T21:21:37.748090+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":8906,"log_dups_size":0,"ondisk_log_size":8906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36430441,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":9621504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7774},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580321897,"num_objects":141681,"num_object_clones":0,"num_object_copies":283362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":141681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113176,"num_write_kb":452704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5745,"num_bytes_recovered":23515136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1183952896,"data_stored":1183879837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141762,"ondisk_log_size":141762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635844,"kb_used_data":393140,"kb_used_omap":4,"kb_used_meta":242683,"kb_avail":93735996,"statfs":{"total":96636764160,"available":95985659904,"internally_reserved":0,"allocated":402575360,"data_stored":402309515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":248508077},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785080,"kb_used_data":478568,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93586760,"statfs":{"total":96636764160,"available":95832842240,"internally_reserved":0,"allocated":490053632,"data_stored":489787787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626470,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514048,"kb_used_data":283824,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93857792,"statfs":{"total":96636764160,"available":96110379008,"internally_reserved":0,"allocated":290635776,"data_stored":290307309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":235728556},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489721856,"data_stored":489718377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291401728,"data_stored":291335627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402829312,"data_stored":402825833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:44.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:44.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'8780", 'reported_seq': 8856, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.584190+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:37.584190+0000', 'last_peered': '2024-09-15T21:21:37.584190+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:37.584190+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 8780, 'log_dups_size': 0, 'ondisk_log_size': 8780, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35962880, 'num_objects': 8780, 'num_object_clones': 0, 'num_object_copies': 17560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8780, 'num_write_kb': 35120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5413}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:44.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'8785", 'reported_seq': 8914, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.671033+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:37.671033+0000', 'last_peered': '2024-09-15T21:21:37.671033+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:37.671033+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 8785, 'log_dups_size': 0, 'ondisk_log_size': 8785, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35962880, 'num_objects': 8780, 'num_object_clones': 0, 'num_object_copies': 17560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6210, 'num_write_kb': 24840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5409}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9040", 'reported_seq': 9163, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.681793+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:37.681793+0000', 'last_peered': '2024-09-15T21:21:37.681793+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:37.681793+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9040, 'log_dups_size': 0, 'ondisk_log_size': 9040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37003264, 'num_objects': 9034, 'num_object_clones': 0, 'num_object_copies': 18068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6676, 'num_write_kb': 26704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5563}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8805", 'reported_seq': 8929, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:39.212969+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:39.212969+0000', 'last_peered': '2024-09-15T21:21:39.212969+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:39.212969+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8805, 'log_dups_size': 0, 'ondisk_log_size': 8805, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36036608, 'num_objects': 8798, 'num_object_clones': 0, 'num_object_copies': 17596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7391, 'num_write_kb': 29564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5460}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8650", 'reported_seq': 8773, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.700878+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:37.700878+0000', 'last_peered': '2024-09-15T21:21:37.700878+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:37.700878+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8650, 'log_dups_size': 0, 'ondisk_log_size': 8650, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35397632, 'num_objects': 8642, 'num_object_clones': 0, 'num_object_copies': 17284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6176, 'num_write_kb': 24704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5301}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8746", 'reported_seq': 8855, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:39.081677+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:39.081677+0000', 'last_peered': '2024-09-15T21:21:39.081677+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:39.081677+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8746, 'log_dups_size': 0, 'ondisk_log_size': 8746, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35803136, 'num_objects': 8741, 'num_object_clones': 0, 'num_object_copies': 17482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7297, 'num_write_kb': 29188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 5475}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:44.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'8897", 'reported_seq': 9012, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:39.191262+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:39.191262+0000', 'last_peered': '2024-09-15T21:21:39.191262+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:39.191262+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 8897, 'log_dups_size': 0, 'ondisk_log_size': 8897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36417536, 'num_objects': 8891, 'num_object_clones': 0, 'num_object_copies': 17782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7296, 'num_write_kb': 29184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 5541}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'8925", 'reported_seq': 9051, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.709180+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:37.709180+0000', 'last_peered': '2024-09-15T21:21:37.709180+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:37.709180+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 8925, 'log_dups_size': 0, 'ondisk_log_size': 8925, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36548608, 'num_objects': 8923, 'num_object_clones': 0, 'num_object_copies': 17846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6232, 'num_write_kb': 24928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5490}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'8834", 'reported_seq': 8962, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.743976+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:37.743976+0000', 'last_peered': '2024-09-15T21:21:37.743976+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:37.743976+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 8834, 'log_dups_size': 0, 'ondisk_log_size': 8834, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36151296, 'num_objects': 8826, 'num_object_clones': 0, 'num_object_copies': 17652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6419, 'num_write_kb': 25676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5447}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'8797", 'reported_seq': 8920, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:37.664170+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:37.664170+0000', 'last_peered': '2024-09-15T21:21:37.664170+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:37.664170+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 8797, 'log_dups_size': 0, 'ondisk_log_size': 8797, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36016128, 'num_objects': 8793, 'num_object_clones': 0, 'num_object_copies': 17586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6444, 'num_write_kb': 25776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5428}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:44.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'8906", 'reported_seq': 13739, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:37.748090+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:37.748090+0000', 'last_peered': '2024-09-15T21:21:37.748090+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:37.748090+0000', 'last_undegraded': '2024-09-15T21:21:37.748090+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 8906, 'log_dups_size': 0, 'ondisk_log_size': 8906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36430441, 'num_objects': 8895, 'num_object_clones': 0, 'num_object_copies': 17790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1121, 'num_objects_unfound': 0, 'num_objects_dirty': 8895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6566, 'num_write_kb': 26264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2352, 'num_bytes_recovered': 9621504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7774}, {'shards': '0,2', 'objects': 1121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:44.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:44.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 231 20 144322 144302 2.43919 0.785156 0.191167 0.0319439 2024-09-15T21:21:44.998 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:44.999 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/288034 objects degraded (11.700%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":318,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144017,"data_bytes":589881980,"bytes_used":1988407296,"bytes_avail":287921885184,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":288034,"degraded_ratio":0.11699660456751633,"misplaced_objects":945,"misplaced_total":288034,"misplaced_ratio":0.0032808626759340907,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":184579,"recovering_keys_per_sec":0,"num_objects_recovered":541,"num_bytes_recovered":2215936,"num_keys_recovered":0,"write_bytes_sec":2274322,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:45.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:45.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:45.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 232 20 144749 144729 2.43586 1.66797 0.0382347 0.0320639 2024-09-15T21:21:45.890 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:45.931 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-15T21:21:44.394200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":589881980,"num_objects":144017,"num_object_clones":0,"num_object_copies":288034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":945,"num_objects_unfound":0,"num_objects_dirty":144017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115512,"num_write_kb":462042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5922,"num_bytes_recovered":24236051,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144099,"ondisk_log_size":144099,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1941804,"kb_used_data":1174844,"kb_used_omap":12,"kb_used_meta":766899,"kb_avail":281173716,"statfs":{"total":289910292480,"available":287921885184,"internally_reserved":0,"allocated":1203040256,"data_stored":1202180099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":785304586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,9,16,17,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27303936,"num_objects":6666,"num_object_clones":0,"num_object_copies":13332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-541,"num_objects_unfound":0,"num_objects_dirty":6666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6666,"num_write_kb":26664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":541,"num_bytes_recovered":2215936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005306"},"pg_stats":[{"pgid":"2.f","version":"46'8950","reported_seq":9025,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.794713+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:42.794713+0000","last_peered":"2024-09-15T21:21:42.794713+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:42.794713+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":8950,"log_dups_size":0,"ondisk_log_size":8950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8950,"num_write_kb":35800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5583},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9077","reported_seq":9173,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.980345+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:43.980345+0000","last_peered":"2024-09-15T21:21:43.980345+0000","last_clean":"2024-09-15T21:21:43.980345+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:43.980345+0000","last_undegraded":"2024-09-15T21:21:43.980345+0000","last_fullsized":"2024-09-15T21:21:43.980345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9141","reported_seq":16056,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:42.774490+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:42.774490+0000","last_peered":"2024-09-15T21:21:42.774490+0000","last_clean":"2024-09-15T21:21:42.774490+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:42.774490+0000","last_undegraded":"2024-09-15T21:21:42.774490+0000","last_fullsized":"2024-09-15T21:21:42.774490+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9141,"log_dups_size":0,"ondisk_log_size":9141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37425152,"num_objects":9137,"num_object_clones":0,"num_object_copies":18274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7864,"num_write_kb":31456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9071","reported_seq":9140,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.971434+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:43.971434+0000","last_peered":"2024-09-15T21:21:43.971434+0000","last_clean":"2024-09-15T21:21:43.971434+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:43.971434+0000","last_undegraded":"2024-09-15T21:21:43.971434+0000","last_fullsized":"2024-09-15T21:21:43.971434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":36284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9106","reported_seq":9224,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:42.777786+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:42.777786+0000","last_peered":"2024-09-15T21:21:42.777786+0000","last_clean":"2024-09-15T21:21:42.777786+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:42.777786+0000","last_undegraded":"2024-09-15T21:21:42.777786+0000","last_fullsized":"2024-09-15T21:21:42.777786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9106,"log_dups_size":0,"ondisk_log_size":9106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37269504,"num_objects":9099,"num_object_clones":0,"num_object_copies":18198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":27856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'8932","reported_seq":9059,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.748568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:42.748568+0000","last_peered":"2024-09-15T21:21:42.748568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:42.748568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":8932,"log_dups_size":0,"ondisk_log_size":8932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36564992,"num_objects":8927,"num_object_clones":0,"num_object_copies":17854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6357,"num_write_kb":25428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9188","reported_seq":9311,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.727038+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:42.727038+0000","last_peered":"2024-09-15T21:21:42.727038+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:42.727038+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9188,"log_dups_size":0,"ondisk_log_size":9188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37609472,"num_objects":9182,"num_object_clones":0,"num_object_copies":18364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6824,"num_write_kb":27296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5711},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'8948","reported_seq":9068,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:42.792115+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:42.792115+0000","last_peered":"2024-09-15T21:21:42.792115+0000","last_clean":"2024-09-15T21:21:42.792115+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:42.792115+0000","last_undegraded":"2024-09-15T21:21:42.792115+0000","last_fullsized":"2024-09-15T21:21:42.792115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":8948,"log_dups_size":0,"ondisk_log_size":8948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642816,"num_objects":8946,"num_object_clones":0,"num_object_copies":17892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8957","reported_seq":9081,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.987307+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:43.987307+0000","last_peered":"2024-09-15T21:21:43.987307+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:43.987307+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7543,"num_write_kb":30172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8789","reported_seq":8911,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.796069+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:42.796069+0000","last_peered":"2024-09-15T21:21:42.796069+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:42.796069+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8789,"log_dups_size":0,"ondisk_log_size":8789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35966976,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5440},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8877","reported_seq":8986,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.979291+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:43.979291+0000","last_peered":"2024-09-15T21:21:43.979291+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:43.979291+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36339712,"num_objects":8872,"num_object_clones":0,"num_object_copies":17744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9040","reported_seq":9155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.956599+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:43.956599+0000","last_peered":"2024-09-15T21:21:43.956599+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:43.956599+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":29756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9082","reported_seq":9208,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.817977+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:42.817977+0000","last_peered":"2024-09-15T21:21:42.817977+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:42.817977+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9082,"log_dups_size":0,"ondisk_log_size":9082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191680,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5647},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'8973","reported_seq":9102,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.777783+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:42.777783+0000","last_peered":"2024-09-15T21:21:42.777783+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:42.777783+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36720640,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6558,"num_write_kb":26232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5586},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'8919","reported_seq":9042,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:42.778124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:42.778124+0000","last_peered":"2024-09-15T21:21:42.778124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:42.778124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":8919,"log_dups_size":0,"ondisk_log_size":8919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6566,"num_write_kb":26264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5550},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9046","reported_seq":14231,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:42.774599+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:42.774599+0000","last_peered":"2024-09-15T21:21:42.774599+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:42.774599+0000","last_undegraded":"2024-09-15T21:21:42.774599+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9046,"log_dups_size":0,"ondisk_log_size":9046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003881,"num_objects":9035,"num_object_clones":0,"num_object_copies":18070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":945,"num_objects_unfound":0,"num_objects_dirty":9035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":26824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2528,"num_bytes_recovered":10342400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8090},{"shards":"0,2","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":589881961,"num_objects":144015,"num_object_clones":0,"num_object_copies":288030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":945,"num_objects_unfound":0,"num_objects_dirty":144015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115510,"num_write_kb":462040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5921,"num_bytes_recovered":24236032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1203728384,"data_stored":1203655325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144096,"ondisk_log_size":144096,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628028,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93743812,"statfs":{"total":96636764160,"available":95993663488,"internally_reserved":0,"allocated":407351296,"data_stored":407085451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783708,"kb_used_data":485516,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93588132,"statfs":{"total":96636764160,"available":95834247168,"internally_reserved":0,"allocated":497168384,"data_stored":496902539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":305327793},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530068,"kb_used_data":291524,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93841772,"statfs":{"total":96636764160,"available":96093974528,"internally_reserved":0,"allocated":298520576,"data_stored":298192109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":244248236},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496877568,"data_stored":496874089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299462656,"data_stored":299396555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407388160,"data_stored":407384681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:46.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 233 20 145223 145203 2.43336 1.85156 0.0137034 0.0320984 2024-09-15T21:21:47.561 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 234 20 145665 145645 2.43034 1.72656 0.00786334 0.0321263 2024-09-15T21:21:48.562 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 235 20 146098 146078 2.42718 1.69141 0.0144983 0.0321778 2024-09-15T21:21:48.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:49.562 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 236 20 146562 146542 2.42458 1.8125 0.0623834 0.0322142 2024-09-15T21:21:49.702 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:49.705 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:49.802 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-15T21:21:48.395131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595116668,"num_objects":145295,"num_object_clones":0,"num_object_copies":290590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116790,"num_write_kb":467154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":25436179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145377,"ondisk_log_size":145377,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1964000,"kb_used_data":1187312,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281151520,"statfs":{"total":289910292480,"available":287899156480,"internally_reserved":0,"allocated":1215807488,"data_stored":1214947331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":795266058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,8,9,19,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-632,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2588672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005302"},"pg_stats":[{"pgid":"2.f","version":"46'9085","reported_seq":9161,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.870127+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:47.870127+0000","last_peered":"2024-09-15T21:21:47.870127+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:47.870127+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9085,"log_dups_size":0,"ondisk_log_size":9085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":36340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9077","reported_seq":9173,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.980345+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:43.980345+0000","last_peered":"2024-09-15T21:21:43.980345+0000","last_clean":"2024-09-15T21:21:43.980345+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:43.980345+0000","last_undegraded":"2024-09-15T21:21:43.980345+0000","last_fullsized":"2024-09-15T21:21:43.980345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9267","reported_seq":16181,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.807723+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:47.807723+0000","last_peered":"2024-09-15T21:21:47.807723+0000","last_clean":"2024-09-15T21:21:47.807723+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:47.807723+0000","last_undegraded":"2024-09-15T21:21:47.807723+0000","last_fullsized":"2024-09-15T21:21:47.807723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9267,"log_dups_size":0,"ondisk_log_size":9267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9071","reported_seq":9140,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.971434+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:43.971434+0000","last_peered":"2024-09-15T21:21:43.971434+0000","last_clean":"2024-09-15T21:21:43.971434+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:43.971434+0000","last_undegraded":"2024-09-15T21:21:43.971434+0000","last_fullsized":"2024-09-15T21:21:43.971434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":36284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9225","reported_seq":9342,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.855653+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:47.855653+0000","last_peered":"2024-09-15T21:21:47.855653+0000","last_clean":"2024-09-15T21:21:47.855653+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:47.855653+0000","last_undegraded":"2024-09-15T21:21:47.855653+0000","last_fullsized":"2024-09-15T21:21:47.855653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9045","reported_seq":9174,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.789495+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:47.789495+0000","last_peered":"2024-09-15T21:21:47.789495+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:47.789495+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37027840,"num_objects":9040,"num_object_clones":0,"num_object_copies":18080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6470,"num_write_kb":25880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5669},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9308","reported_seq":9431,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.795479+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:47.795479+0000","last_peered":"2024-09-15T21:21:47.795479+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:47.795479+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9308,"log_dups_size":0,"ondisk_log_size":9308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38100992,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5831},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9054","reported_seq":9174,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.700406+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:47.700406+0000","last_peered":"2024-09-15T21:21:47.700406+0000","last_clean":"2024-09-15T21:21:47.700406+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:47.700406+0000","last_undegraded":"2024-09-15T21:21:47.700406+0000","last_fullsized":"2024-09-15T21:21:47.700406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9054,"log_dups_size":0,"ondisk_log_size":9054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37076992,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":27356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8957","reported_seq":9081,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.987307+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:43.987307+0000","last_peered":"2024-09-15T21:21:43.987307+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:43.987307+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7543,"num_write_kb":30172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8877","reported_seq":9000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.701858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:47.701858+0000","last_peered":"2024-09-15T21:21:47.701858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:47.701858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327424,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6403,"num_write_kb":25612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5528},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8877","reported_seq":8986,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.979291+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:43.979291+0000","last_peered":"2024-09-15T21:21:43.979291+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:43.979291+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36339712,"num_objects":8872,"num_object_clones":0,"num_object_copies":17744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9040","reported_seq":9155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.956599+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:43.956599+0000","last_peered":"2024-09-15T21:21:43.956599+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:43.956599+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":29756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9203","reported_seq":9331,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.867132+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:47.867132+0000","last_peered":"2024-09-15T21:21:47.867132+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:47.867132+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9203,"log_dups_size":0,"ondisk_log_size":9203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687296,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5768},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9075","reported_seq":9204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.863339+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:47.863339+0000","last_peered":"2024-09-15T21:21:47.863339+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:47.863339+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9075,"log_dups_size":0,"ondisk_log_size":9075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6660,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5688},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9048","reported_seq":9172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.807135+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:47.807135+0000","last_peered":"2024-09-15T21:21:47.807135+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:47.807135+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9048,"log_dups_size":0,"ondisk_log_size":9048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5679},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9165","reported_seq":14937,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:47.852018+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:47.852018+0000","last_peered":"2024-09-15T21:21:47.852018+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:47.852018+0000","last_undegraded":"2024-09-15T21:21:47.852018+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9165,"log_dups_size":0,"ondisk_log_size":9165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37491305,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2821,"num_bytes_recovered":11542528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8502},{"shards":"0,2","objects":652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595116649,"num_objects":145293,"num_object_clones":0,"num_object_copies":290586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116788,"num_write_kb":467152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6214,"num_bytes_recovered":25436160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1216557056,"data_stored":1216483997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145374,"ondisk_log_size":145374,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628028,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93743812,"statfs":{"total":96636764160,"available":95993663488,"internally_reserved":0,"allocated":407351296,"data_stored":407085451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793456,"kb_used_data":491104,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93578384,"statfs":{"total":96636764160,"available":95824265216,"internally_reserved":0,"allocated":502890496,"data_stored":502624651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542516,"kb_used_data":298404,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93829324,"statfs":{"total":96636764160,"available":96081227776,"internally_reserved":0,"allocated":305565696,"data_stored":305237229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":249949868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502599680,"data_stored":502596201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306569216,"data_stored":306503115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407388160,"data_stored":407384681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:49.804 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:50.277 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:50.278 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:50.293 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-15T21:21:48.395131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595116668,"num_objects":145295,"num_object_clones":0,"num_object_copies":290590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116790,"num_write_kb":467154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":25436179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145377,"ondisk_log_size":145377,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1964000,"kb_used_data":1187312,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281151520,"statfs":{"total":289910292480,"available":287899156480,"internally_reserved":0,"allocated":1215807488,"data_stored":1214947331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":795266058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,8,9,19,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-632,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2588672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005302"},"pg_stats":[{"pgid":"2.f","version":"46'9085","reported_seq":9161,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.870127+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:47.870127+0000","last_peered":"2024-09-15T21:21:47.870127+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:47.870127+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9085,"log_dups_size":0,"ondisk_log_size":9085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":36340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9077","reported_seq":9173,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.980345+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:43.980345+0000","last_peered":"2024-09-15T21:21:43.980345+0000","last_clean":"2024-09-15T21:21:43.980345+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:43.980345+0000","last_undegraded":"2024-09-15T21:21:43.980345+0000","last_fullsized":"2024-09-15T21:21:43.980345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9267","reported_seq":16181,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.807723+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:47.807723+0000","last_peered":"2024-09-15T21:21:47.807723+0000","last_clean":"2024-09-15T21:21:47.807723+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:47.807723+0000","last_undegraded":"2024-09-15T21:21:47.807723+0000","last_fullsized":"2024-09-15T21:21:47.807723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9267,"log_dups_size":0,"ondisk_log_size":9267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9071","reported_seq":9140,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.971434+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:43.971434+0000","last_peered":"2024-09-15T21:21:43.971434+0000","last_clean":"2024-09-15T21:21:43.971434+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:43.971434+0000","last_undegraded":"2024-09-15T21:21:43.971434+0000","last_fullsized":"2024-09-15T21:21:43.971434+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37154816,"num_objects":9071,"num_object_clones":0,"num_object_copies":18142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9071,"num_write_kb":36284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9225","reported_seq":9342,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.855653+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:47.855653+0000","last_peered":"2024-09-15T21:21:47.855653+0000","last_clean":"2024-09-15T21:21:47.855653+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:47.855653+0000","last_undegraded":"2024-09-15T21:21:47.855653+0000","last_fullsized":"2024-09-15T21:21:47.855653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9045","reported_seq":9174,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.789495+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:47.789495+0000","last_peered":"2024-09-15T21:21:47.789495+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:47.789495+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37027840,"num_objects":9040,"num_object_clones":0,"num_object_copies":18080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6470,"num_write_kb":25880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5669},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9308","reported_seq":9431,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.795479+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:47.795479+0000","last_peered":"2024-09-15T21:21:47.795479+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:47.795479+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9308,"log_dups_size":0,"ondisk_log_size":9308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38100992,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5831},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9054","reported_seq":9174,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.700406+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:47.700406+0000","last_peered":"2024-09-15T21:21:47.700406+0000","last_clean":"2024-09-15T21:21:47.700406+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:47.700406+0000","last_undegraded":"2024-09-15T21:21:47.700406+0000","last_fullsized":"2024-09-15T21:21:47.700406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9054,"log_dups_size":0,"ondisk_log_size":9054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37076992,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":27356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'8957","reported_seq":9081,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.987307+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:43.987307+0000","last_peered":"2024-09-15T21:21:43.987307+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:43.987307+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7543,"num_write_kb":30172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436756+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436756+0000","last_peered":"2024-09-15T21:20:16.436756+0000","last_clean":"2024-09-15T21:20:16.436756+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436756+0000","last_undegraded":"2024-09-15T21:20:16.436756+0000","last_fullsized":"2024-09-15T21:20:16.436756+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8877","reported_seq":9000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.701858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:47.701858+0000","last_peered":"2024-09-15T21:21:47.701858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:47.701858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327424,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6403,"num_write_kb":25612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5528},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'8877","reported_seq":8986,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.979291+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:43.979291+0000","last_peered":"2024-09-15T21:21:43.979291+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:43.979291+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36339712,"num_objects":8872,"num_object_clones":0,"num_object_copies":17744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9040","reported_seq":9155,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.956599+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:43.956599+0000","last_peered":"2024-09-15T21:21:43.956599+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:43.956599+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37003264,"num_objects":9034,"num_object_clones":0,"num_object_copies":18068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7439,"num_write_kb":29756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9203","reported_seq":9331,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.867132+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:47.867132+0000","last_peered":"2024-09-15T21:21:47.867132+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:47.867132+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9203,"log_dups_size":0,"ondisk_log_size":9203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687296,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5768},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9075","reported_seq":9204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.863339+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:47.863339+0000","last_peered":"2024-09-15T21:21:47.863339+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:47.863339+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9075,"log_dups_size":0,"ondisk_log_size":9075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6660,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5688},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437635+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437635+0000","last_peered":"2024-09-15T21:20:16.437635+0000","last_clean":"2024-09-15T21:20:16.437635+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437635+0000","last_undegraded":"2024-09-15T21:20:16.437635+0000","last_fullsized":"2024-09-15T21:20:16.437635+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9048","reported_seq":9172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.807135+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:47.807135+0000","last_peered":"2024-09-15T21:21:47.807135+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:47.807135+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9048,"log_dups_size":0,"ondisk_log_size":9048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5679},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9165","reported_seq":14937,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:47.852018+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:47.852018+0000","last_peered":"2024-09-15T21:21:47.852018+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:47.852018+0000","last_undegraded":"2024-09-15T21:21:47.852018+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9165,"log_dups_size":0,"ondisk_log_size":9165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37491305,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2821,"num_bytes_recovered":11542528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8502},{"shards":"0,2","objects":652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595116649,"num_objects":145293,"num_object_clones":0,"num_object_copies":290586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116788,"num_write_kb":467152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6214,"num_bytes_recovered":25436160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1216557056,"data_stored":1216483997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145374,"ondisk_log_size":145374,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628028,"kb_used_data":397804,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93743812,"statfs":{"total":96636764160,"available":95993663488,"internally_reserved":0,"allocated":407351296,"data_stored":407085451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793456,"kb_used_data":491104,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93578384,"statfs":{"total":96636764160,"available":95824265216,"internally_reserved":0,"allocated":502890496,"data_stored":502624651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542516,"kb_used_data":298404,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93829324,"statfs":{"total":96636764160,"available":96081227776,"internally_reserved":0,"allocated":305565696,"data_stored":305237229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":249949868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502599680,"data_stored":502596201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306569216,"data_stored":306503115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":407388160,"data_stored":407384681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9085", 'reported_seq': 9161, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.870127+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:47.870127+0000', 'last_peered': '2024-09-15T21:21:47.870127+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:47.870127+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9085, 'log_dups_size': 0, 'ondisk_log_size': 9085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37212160, 'num_objects': 9085, 'num_object_clones': 0, 'num_object_copies': 18170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9085, 'num_write_kb': 36340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5718}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9045", 'reported_seq': 9174, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.789495+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:47.789495+0000', 'last_peered': '2024-09-15T21:21:47.789495+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:47.789495+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9045, 'log_dups_size': 0, 'ondisk_log_size': 9045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37027840, 'num_objects': 9040, 'num_object_clones': 0, 'num_object_copies': 18080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6470, 'num_write_kb': 25880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5669}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9308", 'reported_seq': 9431, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.795479+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:47.795479+0000', 'last_peered': '2024-09-15T21:21:47.795479+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:47.795479+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9308, 'log_dups_size': 0, 'ondisk_log_size': 9308, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38100992, 'num_objects': 9302, 'num_object_clones': 0, 'num_object_copies': 18604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6944, 'num_write_kb': 27776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5831}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'8957", 'reported_seq': 9081, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.987307+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:43.987307+0000', 'last_peered': '2024-09-15T21:21:43.987307+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:43.987307+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 8957, 'log_dups_size': 0, 'ondisk_log_size': 8957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36659200, 'num_objects': 8950, 'num_object_clones': 0, 'num_object_copies': 17900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7543, 'num_write_kb': 30172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:50.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'8877", 'reported_seq': 9000, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.701858+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:47.701858+0000', 'last_peered': '2024-09-15T21:21:47.701858+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:47.701858+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 8877, 'log_dups_size': 0, 'ondisk_log_size': 8877, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36327424, 'num_objects': 8869, 'num_object_clones': 0, 'num_object_copies': 17738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6403, 'num_write_kb': 25612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5528}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'8877", 'reported_seq': 8986, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.979291+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:43.979291+0000', 'last_peered': '2024-09-15T21:21:43.979291+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:43.979291+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 8877, 'log_dups_size': 0, 'ondisk_log_size': 8877, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36339712, 'num_objects': 8872, 'num_object_clones': 0, 'num_object_copies': 17744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7428, 'num_write_kb': 29712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3266}, {'shards': '2', 'objects': 5606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9040", 'reported_seq': 9155, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.956599+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:43.956599+0000', 'last_peered': '2024-09-15T21:21:43.956599+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:43.956599+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9040, 'log_dups_size': 0, 'ondisk_log_size': 9040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37003264, 'num_objects': 9034, 'num_object_clones': 0, 'num_object_copies': 18068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7439, 'num_write_kb': 29756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 5684}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9203", 'reported_seq': 9331, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.867132+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:47.867132+0000', 'last_peered': '2024-09-15T21:21:47.867132+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:47.867132+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9203, 'log_dups_size': 0, 'ondisk_log_size': 9203, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37687296, 'num_objects': 9201, 'num_object_clones': 0, 'num_object_copies': 18402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6510, 'num_write_kb': 26040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5768}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9075", 'reported_seq': 9204, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.863339+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:47.863339+0000', 'last_peered': '2024-09-15T21:21:47.863339+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:47.863339+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9075, 'log_dups_size': 0, 'ondisk_log_size': 9075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37138432, 'num_objects': 9067, 'num_object_clones': 0, 'num_object_copies': 18134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6660, 'num_write_kb': 26640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5688}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:50.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9048", 'reported_seq': 9172, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:47.807135+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:47.807135+0000', 'last_peered': '2024-09-15T21:21:47.807135+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:47.807135+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9048, 'log_dups_size': 0, 'ondisk_log_size': 9048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37044224, 'num_objects': 9044, 'num_object_clones': 0, 'num_object_copies': 18088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6695, 'num_write_kb': 26780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5679}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:50.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9165", 'reported_seq': 14937, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:47.852018+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:47.852018+0000', 'last_peered': '2024-09-15T21:21:47.852018+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:47.852018+0000', 'last_undegraded': '2024-09-15T21:21:47.852018+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 9165, 'log_dups_size': 0, 'ondisk_log_size': 9165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37491305, 'num_objects': 9154, 'num_object_clones': 0, 'num_object_copies': 18308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 652, 'num_objects_unfound': 0, 'num_objects_dirty': 9154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6825, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2821, 'num_bytes_recovered': 11542528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8502}, {'shards': '0,2', 'objects': 652}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:50.562 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 237 20 146904 146884 2.41999 1.33594 0.0192948 0.0322725 2024-09-15T21:21:50.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:21:50.776+0000 7f3677fff640 0 --2- 172.21.15.114:0/3679011976 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f368006f320 0x7f3680092fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:21:51.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:51.121 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/291936 objects degraded (11.543%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145968,"data_bytes":597873276,"bytes_used":2019479552,"bytes_avail":287890812928,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":291936,"degraded_ratio":0.11543283459388359,"misplaced_objects":652,"misplaced_total":291936,"misplaced_ratio":0.0022333662172530967,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":160013,"recovering_keys_per_sec":0,"num_objects_recovered":469,"num_bytes_recovered":1921024,"num_keys_recovered":0,"write_bytes_sec":1712727,"read_op_per_sec":0,"write_op_per_sec":418},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:51.195 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:51.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:51.562 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 238 20 147266 147246 2.41576 1.41406 0.0609471 0.032333 2024-09-15T21:21:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:52.029 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:52.063 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-15T21:21:50.395633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597873276,"num_objects":145968,"num_object_clones":0,"num_object_copies":291936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117463,"num_write_kb":469846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6215,"num_bytes_recovered":25436179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146050,"ondisk_log_size":146050,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1972148,"kb_used_data":1191300,"kb_used_omap":12,"kb_used_meta":780787,"kb_avail":281143372,"statfs":{"total":289910292480,"available":287890812928,"internally_reserved":0,"allocated":1219891200,"data_stored":1219031043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":799525898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,7,6,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20561920,"num_objects":5020,"num_object_clones":0,"num_object_copies":10040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-469,"num_objects_unfound":0,"num_objects_dirty":5020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5020,"num_write_kb":20080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":469,"num_bytes_recovered":1921024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005366"},"pg_stats":[{"pgid":"2.f","version":"46'9085","reported_seq":9161,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.870127+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:47.870127+0000","last_peered":"2024-09-15T21:21:47.870127+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:47.870127+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9085,"log_dups_size":0,"ondisk_log_size":9085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212160,"num_objects":9085,"num_object_clones":0,"num_object_copies":18170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9085,"num_write_kb":36340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5718},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9212","reported_seq":9308,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:49.218396+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:49.218396+0000","last_peered":"2024-09-15T21:21:49.218396+0000","last_clean":"2024-09-15T21:21:49.218396+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:49.218396+0000","last_undegraded":"2024-09-15T21:21:49.218396+0000","last_fullsized":"2024-09-15T21:21:49.218396+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9212,"log_dups_size":0,"ondisk_log_size":9212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":31752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9267","reported_seq":16181,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.807723+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:47.807723+0000","last_peered":"2024-09-15T21:21:47.807723+0000","last_clean":"2024-09-15T21:21:47.807723+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:47.807723+0000","last_undegraded":"2024-09-15T21:21:47.807723+0000","last_fullsized":"2024-09-15T21:21:47.807723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9267,"log_dups_size":0,"ondisk_log_size":9267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37941248,"num_objects":9263,"num_object_clones":0,"num_object_copies":18526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7990,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9195","reported_seq":9264,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:49.192825+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:49.192825+0000","last_peered":"2024-09-15T21:21:49.192825+0000","last_clean":"2024-09-15T21:21:49.192825+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:49.192825+0000","last_undegraded":"2024-09-15T21:21:49.192825+0000","last_fullsized":"2024-09-15T21:21:49.192825+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662720,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9195,"num_write_kb":36780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9225","reported_seq":9342,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.855653+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:47.855653+0000","last_peered":"2024-09-15T21:21:47.855653+0000","last_clean":"2024-09-15T21:21:47.855653+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:47.855653+0000","last_undegraded":"2024-09-15T21:21:47.855653+0000","last_fullsized":"2024-09-15T21:21:47.855653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9225,"log_dups_size":0,"ondisk_log_size":9225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9045","reported_seq":9174,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.789495+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:47.789495+0000","last_peered":"2024-09-15T21:21:47.789495+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:47.789495+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9045,"log_dups_size":0,"ondisk_log_size":9045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37027840,"num_objects":9040,"num_object_clones":0,"num_object_copies":18080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6470,"num_write_kb":25880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5669},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9308","reported_seq":9431,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.795479+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:47.795479+0000","last_peered":"2024-09-15T21:21:47.795479+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:47.795479+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9308,"log_dups_size":0,"ondisk_log_size":9308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38100992,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":27776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5831},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9054","reported_seq":9174,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:47.700406+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:47.700406+0000","last_peered":"2024-09-15T21:21:47.700406+0000","last_clean":"2024-09-15T21:21:47.700406+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:47.700406+0000","last_undegraded":"2024-09-15T21:21:47.700406+0000","last_fullsized":"2024-09-15T21:21:47.700406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9054,"log_dups_size":0,"ondisk_log_size":9054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37076992,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":27356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9087","reported_seq":9211,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:49.240541+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:49.240541+0000","last_peered":"2024-09-15T21:21:49.240541+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:49.240541+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9087,"log_dups_size":0,"ondisk_log_size":9087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191680,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'8877","reported_seq":9000,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.701858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:47.701858+0000","last_peered":"2024-09-15T21:21:47.701858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:47.701858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":8877,"log_dups_size":0,"ondisk_log_size":8877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327424,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6403,"num_write_kb":25612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5528},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9027","reported_seq":9135,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:49.238984+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:49.238984+0000","last_peered":"2024-09-15T21:21:49.238984+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:49.238984+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9027,"log_dups_size":0,"ondisk_log_size":9027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36954112,"num_objects":9022,"num_object_clones":0,"num_object_copies":18044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7578,"num_write_kb":30312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3266},{"shards":"2","objects":5756}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9174","reported_seq":9289,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:49.208987+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:49.208987+0000","last_peered":"2024-09-15T21:21:49.208987+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:49.208987+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9174,"log_dups_size":0,"ondisk_log_size":9174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37552128,"num_objects":9168,"num_object_clones":0,"num_object_copies":18336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7573,"num_write_kb":30292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":5818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9203","reported_seq":9331,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.867132+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:47.867132+0000","last_peered":"2024-09-15T21:21:47.867132+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:47.867132+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9203,"log_dups_size":0,"ondisk_log_size":9203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687296,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6510,"num_write_kb":26040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5768},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9075","reported_seq":9204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.863339+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:47.863339+0000","last_peered":"2024-09-15T21:21:47.863339+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:47.863339+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9075,"log_dups_size":0,"ondisk_log_size":9075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6660,"num_write_kb":26640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5688},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9048","reported_seq":9172,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:47.807135+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:47.807135+0000","last_peered":"2024-09-15T21:21:47.807135+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:47.807135+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9048,"log_dups_size":0,"ondisk_log_size":9048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6695,"num_write_kb":26780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5679},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9165","reported_seq":14937,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:47.852018+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:47.852018+0000","last_peered":"2024-09-15T21:21:47.852018+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:47.852018+0000","last_undegraded":"2024-09-15T21:21:47.852018+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9165,"log_dups_size":0,"ondisk_log_size":9165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37491305,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2821,"num_bytes_recovered":11542528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8502},{"shards":"0,2","objects":652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597873257,"num_objects":145966,"num_object_clones":0,"num_object_copies":291932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":652,"num_objects_unfound":0,"num_objects_dirty":145966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117461,"num_write_kb":469844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6214,"num_bytes_recovered":25436160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1220931584,"data_stored":1220858525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146047,"ondisk_log_size":146047,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953527,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636176,"kb_used_data":401792,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93735664,"statfs":{"total":96636764160,"available":95985319936,"internally_reserved":0,"allocated":411435008,"data_stored":411169163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":239988397},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793456,"kb_used_data":491104,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93578384,"statfs":{"total":96636764160,"available":95824265216,"internally_reserved":0,"allocated":502890496,"data_stored":502624651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542516,"kb_used_data":298404,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93829324,"statfs":{"total":96636764160,"available":96081227776,"internally_reserved":0,"allocated":305565696,"data_stored":305237229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":249949868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502599680,"data_stored":502596201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306569216,"data_stored":306503115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411762688,"data_stored":411759209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:52.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout: 239 20 147710 147690 2.41289 1.73438 0.0184967 0.0323671 2024-09-15T21:21:53.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Total time run: 240.024 2024-09-15T21:21:53.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Total writes made: 148140 2024-09-15T21:21:53.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Write size: 4096 2024-09-15T21:21:53.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Object size: 4096 2024-09-15T21:21:53.564 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Bandwidth (MB/sec): 2.41089 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Stddev Bandwidth: 1.45859 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Max bandwidth (MB/sec): 10.9883 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Min bandwidth (MB/sec): 0 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Average IOPS: 617 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Stddev IOPS: 373.404 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Max IOPS: 2813 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Min IOPS: 0 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Average Latency(s): 0.0323999 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Stddev Latency(s): 0.0721862 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Max latency(s): 4.27752 2024-09-15T21:21:53.565 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Min latency(s): 0.00128075 2024-09-15T21:21:53.569 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T21:21:55.065 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:55.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:55.745 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:55.808 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-15T21:21:54.396632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605565564,"num_objects":147846,"num_object_clones":0,"num_object_copies":295692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119511,"num_write_kb":477698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6471,"num_bytes_recovered":26484755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148098,"ondisk_log_size":148098,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2007824,"kb_used_data":1210336,"kb_used_omap":12,"kb_used_meta":797427,"kb_avail":281107696,"statfs":{"total":289910292480,"available":287854280704,"internally_reserved":0,"allocated":1239384064,"data_stored":1238523907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":816565258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,12,1,10,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-725,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":24992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":725,"num_bytes_recovered":2969600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002878"},"pg_stats":[{"pgid":"2.f","version":"46'9241","reported_seq":9314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.864942+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:52.864942+0000","last_peered":"2024-09-15T21:21:52.864942+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:52.864942+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37851136,"num_objects":9241,"num_object_clones":0,"num_object_copies":18482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":36964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9342","reported_seq":9438,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.121545+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:54.121545+0000","last_peered":"2024-09-15T21:21:54.121545+0000","last_clean":"2024-09-15T21:21:54.121545+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:54.121545+0000","last_undegraded":"2024-09-15T21:21:54.121545+0000","last_fullsized":"2024-09-15T21:21:54.121545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9404","reported_seq":16319,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.853877+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:52.853877+0000","last_peered":"2024-09-15T21:21:52.853877+0000","last_clean":"2024-09-15T21:21:52.853877+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:52.853877+0000","last_undegraded":"2024-09-15T21:21:52.853877+0000","last_fullsized":"2024-09-15T21:21:52.853877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38502400,"num_objects":9400,"num_object_clones":0,"num_object_copies":18800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":32508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9303","reported_seq":9372,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.160001+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:54.160001+0000","last_peered":"2024-09-15T21:21:54.160001+0000","last_clean":"2024-09-15T21:21:54.160001+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:54.160001+0000","last_undegraded":"2024-09-15T21:21:54.160001+0000","last_fullsized":"2024-09-15T21:21:54.160001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9345","reported_seq":9464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.696217+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:52.696217+0000","last_peered":"2024-09-15T21:21:52.696217+0000","last_clean":"2024-09-15T21:21:52.696217+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:52.696217+0000","last_undegraded":"2024-09-15T21:21:52.696217+0000","last_fullsized":"2024-09-15T21:21:52.696217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9175","reported_seq":9305,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.884119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:52.884119+0000","last_peered":"2024-09-15T21:21:52.884119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:52.884119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9175,"log_dups_size":0,"ondisk_log_size":9175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":26400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5799},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9433","reported_seq":9554,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.797249+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:52.797249+0000","last_peered":"2024-09-15T21:21:52.797249+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:52.797249+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38612992,"num_objects":9427,"num_object_clones":0,"num_object_copies":18854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9183","reported_seq":9303,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.829739+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:52.829739+0000","last_peered":"2024-09-15T21:21:52.829739+0000","last_clean":"2024-09-15T21:21:52.829739+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:52.829739+0000","last_undegraded":"2024-09-15T21:21:52.829739+0000","last_fullsized":"2024-09-15T21:21:52.829739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9204","reported_seq":9328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.144412+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:54.144412+0000","last_peered":"2024-09-15T21:21:54.144412+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:54.144412+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564416,"num_objects":9171,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7790,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3325},{"shards":"2","objects":5833}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9007","reported_seq":9130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.771811+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:52.771811+0000","last_peered":"2024-09-15T21:21:52.771811+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:52.771811+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9007,"log_dups_size":0,"ondisk_log_size":9007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9162","reported_seq":9272,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.076300+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:54.076300+0000","last_peered":"2024-09-15T21:21:54.076300+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:54.076300+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9162,"log_dups_size":0,"ondisk_log_size":9162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3249},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9295","reported_seq":9410,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.170778+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:54.170778+0000","last_peered":"2024-09-15T21:21:54.170778+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:54.170778+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9295,"log_dups_size":0,"ondisk_log_size":9295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900288,"num_objects":9253,"num_object_clones":0,"num_object_copies":18506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3332},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9332","reported_seq":9457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.676812+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:52.676812+0000","last_peered":"2024-09-15T21:21:52.676812+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:52.676812+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9332,"log_dups_size":0,"ondisk_log_size":9332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6639,"num_write_kb":26556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9200","reported_seq":9330,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.866375+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:52.866375+0000","last_peered":"2024-09-15T21:21:52.866375+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:52.866375+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9200,"log_dups_size":0,"ondisk_log_size":9200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37650432,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":27140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9171","reported_seq":9295,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.835820+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:52.835820+0000","last_peered":"2024-09-15T21:21:52.835820+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:52.835820+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6818,"num_write_kb":27272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5802},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9298","reported_seq":15581,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:52.906296+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:52.906296+0000","last_peered":"2024-09-15T21:21:52.906296+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:52.906296+0000","last_undegraded":"2024-09-15T21:21:52.906296+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38036073,"num_objects":9287,"num_object_clones":0,"num_object_copies":18574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":9287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":12591104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8891},{"shards":"0,2","objects":396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605565545,"num_objects":147844,"num_object_clones":0,"num_object_copies":295688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119509,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6470,"num_bytes_recovered":26484736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237811200,"data_stored":1237734069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148095,"ondisk_log_size":148095,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953528,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644460,"kb_used_data":405916,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93727380,"statfs":{"total":96636764160,"available":95976837120,"internally_reserved":0,"allocated":415657984,"data_stored":415392139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803912,"kb_used_data":497400,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567928,"statfs":{"total":96636764160,"available":95813558272,"internally_reserved":0,"allocated":509337600,"data_stored":509071755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,0,3,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559452,"kb_used_data":307020,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812388,"statfs":{"total":96636764160,"available":96063885312,"internally_reserved":0,"allocated":314388480,"data_stored":314060013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":508956672,"data_stored":508953193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314159104,"data_stored":314093003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414695424,"data_stored":414687873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:55.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:56.345 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:56.345 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:56.418 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-15T21:21:54.396632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605565564,"num_objects":147846,"num_object_clones":0,"num_object_copies":295692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119511,"num_write_kb":477698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6471,"num_bytes_recovered":26484755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148098,"ondisk_log_size":148098,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2007824,"kb_used_data":1210336,"kb_used_omap":12,"kb_used_meta":797427,"kb_avail":281107696,"statfs":{"total":289910292480,"available":287854280704,"internally_reserved":0,"allocated":1239384064,"data_stored":1238523907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":816565258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,12,1,10,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-725,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":24992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":725,"num_bytes_recovered":2969600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002878"},"pg_stats":[{"pgid":"2.f","version":"46'9241","reported_seq":9314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.864942+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:52.864942+0000","last_peered":"2024-09-15T21:21:52.864942+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:52.864942+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37851136,"num_objects":9241,"num_object_clones":0,"num_object_copies":18482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":36964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9342","reported_seq":9438,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.121545+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:54.121545+0000","last_peered":"2024-09-15T21:21:54.121545+0000","last_clean":"2024-09-15T21:21:54.121545+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:54.121545+0000","last_undegraded":"2024-09-15T21:21:54.121545+0000","last_fullsized":"2024-09-15T21:21:54.121545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9404","reported_seq":16319,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.853877+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:52.853877+0000","last_peered":"2024-09-15T21:21:52.853877+0000","last_clean":"2024-09-15T21:21:52.853877+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:52.853877+0000","last_undegraded":"2024-09-15T21:21:52.853877+0000","last_fullsized":"2024-09-15T21:21:52.853877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38502400,"num_objects":9400,"num_object_clones":0,"num_object_copies":18800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":32508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9303","reported_seq":9372,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.160001+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:54.160001+0000","last_peered":"2024-09-15T21:21:54.160001+0000","last_clean":"2024-09-15T21:21:54.160001+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:54.160001+0000","last_undegraded":"2024-09-15T21:21:54.160001+0000","last_fullsized":"2024-09-15T21:21:54.160001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9345","reported_seq":9464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.696217+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:52.696217+0000","last_peered":"2024-09-15T21:21:52.696217+0000","last_clean":"2024-09-15T21:21:52.696217+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:52.696217+0000","last_undegraded":"2024-09-15T21:21:52.696217+0000","last_fullsized":"2024-09-15T21:21:52.696217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9175","reported_seq":9305,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.884119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:52.884119+0000","last_peered":"2024-09-15T21:21:52.884119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:52.884119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9175,"log_dups_size":0,"ondisk_log_size":9175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":26400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5799},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9433","reported_seq":9554,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.797249+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:52.797249+0000","last_peered":"2024-09-15T21:21:52.797249+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:52.797249+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38612992,"num_objects":9427,"num_object_clones":0,"num_object_copies":18854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9183","reported_seq":9303,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.829739+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:52.829739+0000","last_peered":"2024-09-15T21:21:52.829739+0000","last_clean":"2024-09-15T21:21:52.829739+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:52.829739+0000","last_undegraded":"2024-09-15T21:21:52.829739+0000","last_fullsized":"2024-09-15T21:21:52.829739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9204","reported_seq":9328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.144412+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:54.144412+0000","last_peered":"2024-09-15T21:21:54.144412+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:54.144412+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564416,"num_objects":9171,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7790,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3325},{"shards":"2","objects":5833}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9007","reported_seq":9130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.771811+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:52.771811+0000","last_peered":"2024-09-15T21:21:52.771811+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:52.771811+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9007,"log_dups_size":0,"ondisk_log_size":9007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9162","reported_seq":9272,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.076300+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:54.076300+0000","last_peered":"2024-09-15T21:21:54.076300+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:54.076300+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9162,"log_dups_size":0,"ondisk_log_size":9162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3249},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9295","reported_seq":9410,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.170778+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:54.170778+0000","last_peered":"2024-09-15T21:21:54.170778+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:54.170778+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9295,"log_dups_size":0,"ondisk_log_size":9295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900288,"num_objects":9253,"num_object_clones":0,"num_object_copies":18506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3332},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9332","reported_seq":9457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.676812+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:52.676812+0000","last_peered":"2024-09-15T21:21:52.676812+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:52.676812+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9332,"log_dups_size":0,"ondisk_log_size":9332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6639,"num_write_kb":26556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9200","reported_seq":9330,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.866375+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:52.866375+0000","last_peered":"2024-09-15T21:21:52.866375+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:52.866375+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9200,"log_dups_size":0,"ondisk_log_size":9200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37650432,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":27140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9171","reported_seq":9295,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.835820+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:52.835820+0000","last_peered":"2024-09-15T21:21:52.835820+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:52.835820+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6818,"num_write_kb":27272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5802},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9298","reported_seq":15581,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:52.906296+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:52.906296+0000","last_peered":"2024-09-15T21:21:52.906296+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:52.906296+0000","last_undegraded":"2024-09-15T21:21:52.906296+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38036073,"num_objects":9287,"num_object_clones":0,"num_object_copies":18574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":9287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":12591104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8891},{"shards":"0,2","objects":396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605565545,"num_objects":147844,"num_object_clones":0,"num_object_copies":295688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119509,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6470,"num_bytes_recovered":26484736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237811200,"data_stored":1237734069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148095,"ondisk_log_size":148095,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953528,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644460,"kb_used_data":405916,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93727380,"statfs":{"total":96636764160,"available":95976837120,"internally_reserved":0,"allocated":415657984,"data_stored":415392139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803912,"kb_used_data":497400,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567928,"statfs":{"total":96636764160,"available":95813558272,"internally_reserved":0,"allocated":509337600,"data_stored":509071755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,0,3,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559452,"kb_used_data":307020,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812388,"statfs":{"total":96636764160,"available":96063885312,"internally_reserved":0,"allocated":314388480,"data_stored":314060013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":508956672,"data_stored":508953193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314159104,"data_stored":314093003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414695424,"data_stored":414687873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:56.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:56.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9241", 'reported_seq': 9314, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.864942+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:52.864942+0000', 'last_peered': '2024-09-15T21:21:52.864942+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:52.864942+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9241, 'log_dups_size': 0, 'ondisk_log_size': 9241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37851136, 'num_objects': 9241, 'num_object_clones': 0, 'num_object_copies': 18482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9241, 'num_write_kb': 36964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5874}, {'shards': '0,2', 'objects': 3367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:56.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9175", 'reported_seq': 9305, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.884119+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:52.884119+0000', 'last_peered': '2024-09-15T21:21:52.884119+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:52.884119+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9175, 'log_dups_size': 0, 'ondisk_log_size': 9175, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37560320, 'num_objects': 9170, 'num_object_clones': 0, 'num_object_copies': 18340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3373, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6600, 'num_write_kb': 26400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5799}, {'shards': '0,2', 'objects': 3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9433", 'reported_seq': 9554, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.797249+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:52.797249+0000', 'last_peered': '2024-09-15T21:21:52.797249+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:52.797249+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9433, 'log_dups_size': 0, 'ondisk_log_size': 9433, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38612992, 'num_objects': 9427, 'num_object_clones': 0, 'num_object_copies': 18854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3473, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7069, 'num_write_kb': 28276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5956}, {'shards': '0,2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'9204", 'reported_seq': 9328, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:54.144412+0000', 'last_change': '2024-09-15T21:18:43.798229+0000', 'last_active': '2024-09-15T21:21:54.144412+0000', 'last_peered': '2024-09-15T21:21:54.144412+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:54.144412+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9204, 'log_dups_size': 0, 'ondisk_log_size': 9204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37564416, 'num_objects': 9171, 'num_object_clones': 0, 'num_object_copies': 18342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9171, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7790, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3325}, {'shards': '2', 'objects': 5833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9007", 'reported_seq': 9130, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.771811+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:52.771811+0000', 'last_peered': '2024-09-15T21:21:52.771811+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:52.771811+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9007, 'log_dups_size': 0, 'ondisk_log_size': 9007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36859904, 'num_objects': 8999, 'num_object_clones': 0, 'num_object_copies': 17998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3343, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6533, 'num_write_kb': 26132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5658}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9162", 'reported_seq': 9272, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:54.076300+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:54.076300+0000', 'last_peered': '2024-09-15T21:21:54.076300+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:54.076300+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9162, 'log_dups_size': 0, 'ondisk_log_size': 9162, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37367808, 'num_objects': 9123, 'num_object_clones': 0, 'num_object_copies': 18246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7713, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3249}, {'shards': '2', 'objects': 5857}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9295", 'reported_seq': 9410, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:54.170778+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:54.170778+0000', 'last_peered': '2024-09-15T21:21:54.170778+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:54.170778+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9295, 'log_dups_size': 0, 'ondisk_log_size': 9295, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37900288, 'num_objects': 9253, 'num_object_clones': 0, 'num_object_copies': 18506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3352, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7694, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3332}, {'shards': '2', 'objects': 5903}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9332", 'reported_seq': 9457, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.676812+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:52.676812+0000', 'last_peered': '2024-09-15T21:21:52.676812+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:52.676812+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9332, 'log_dups_size': 0, 'ondisk_log_size': 9332, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38215680, 'num_objects': 9330, 'num_object_clones': 0, 'num_object_copies': 18660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6639, 'num_write_kb': 26556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5897}, {'shards': '0,2', 'objects': 3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9200", 'reported_seq': 9330, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.866375+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:52.866375+0000', 'last_peered': '2024-09-15T21:21:52.866375+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:52.866375+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9200, 'log_dups_size': 0, 'ondisk_log_size': 9200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37650432, 'num_objects': 9192, 'num_object_clones': 0, 'num_object_copies': 18384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3382, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6785, 'num_write_kb': 27140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5813}, {'shards': '0,2', 'objects': 3379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9171", 'reported_seq': 9295, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:52.835820+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:52.835820+0000', 'last_peered': '2024-09-15T21:21:52.835820+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:52.835820+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9171, 'log_dups_size': 0, 'ondisk_log_size': 9171, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37548032, 'num_objects': 9167, 'num_object_clones': 0, 'num_object_copies': 18334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3367, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6818, 'num_write_kb': 27272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5802}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:56.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9298", 'reported_seq': 15581, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:52.906296+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:52.906296+0000', 'last_peered': '2024-09-15T21:21:52.906296+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:52.906296+0000', 'last_undegraded': '2024-09-15T21:21:52.906296+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 9298, 'log_dups_size': 0, 'ondisk_log_size': 9298, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38036073, 'num_objects': 9287, 'num_object_clones': 0, 'num_object_copies': 18574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 396, 'num_objects_unfound': 0, 'num_objects_dirty': 9287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6958, 'num_write_kb': 27832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3077, 'num_bytes_recovered': 12591104, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8891}, {'shards': '0,2', 'objects': 396}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:57.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:57.414 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33699/295692 objects degraded (11.397%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":330,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147846,"data_bytes":605565564,"bytes_used":2056011776,"bytes_avail":287854280704,"bytes_total":289910292480,"degraded_objects":33699,"degraded_total":295692,"degraded_ratio":0.11396655979870945,"misplaced_objects":396,"misplaced_total":295692,"misplaced_ratio":0.0013392313623635404,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":187346,"recovering_keys_per_sec":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"write_bytes_sec":1335651,"read_op_per_sec":0,"write_op_per_sec":333},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:21:57.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:57.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:57.990 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:21:57.992 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:21:58.028 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-15T21:21:56.397136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605565564,"num_objects":147846,"num_object_clones":0,"num_object_copies":295692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119511,"num_write_kb":477698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6471,"num_bytes_recovered":26484755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148098,"ondisk_log_size":148098,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2007824,"kb_used_data":1210336,"kb_used_omap":12,"kb_used_meta":797427,"kb_avail":281107696,"statfs":{"total":289910292480,"available":287854280704,"internally_reserved":0,"allocated":1239384064,"data_stored":1238523907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":816565258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,12,1,10,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15683584,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-549,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3999,"num_write_kb":15656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002935"},"pg_stats":[{"pgid":"2.f","version":"46'9241","reported_seq":9314,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.864942+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:52.864942+0000","last_peered":"2024-09-15T21:21:52.864942+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:52.864942+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9241,"log_dups_size":0,"ondisk_log_size":9241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37851136,"num_objects":9241,"num_object_clones":0,"num_object_copies":18482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9241,"num_write_kb":36964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":3367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9342","reported_seq":9438,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.121545+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:54.121545+0000","last_peered":"2024-09-15T21:21:54.121545+0000","last_clean":"2024-09-15T21:21:54.121545+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:54.121545+0000","last_undegraded":"2024-09-15T21:21:54.121545+0000","last_fullsized":"2024-09-15T21:21:54.121545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9342,"log_dups_size":0,"ondisk_log_size":9342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8068,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9404","reported_seq":16319,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.853877+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:52.853877+0000","last_peered":"2024-09-15T21:21:52.853877+0000","last_clean":"2024-09-15T21:21:52.853877+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:52.853877+0000","last_undegraded":"2024-09-15T21:21:52.853877+0000","last_fullsized":"2024-09-15T21:21:52.853877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38502400,"num_objects":9400,"num_object_clones":0,"num_object_copies":18800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":32508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9303","reported_seq":9372,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:54.160001+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:54.160001+0000","last_peered":"2024-09-15T21:21:54.160001+0000","last_clean":"2024-09-15T21:21:54.160001+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:54.160001+0000","last_undegraded":"2024-09-15T21:21:54.160001+0000","last_fullsized":"2024-09-15T21:21:54.160001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9345","reported_seq":9464,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.696217+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:52.696217+0000","last_peered":"2024-09-15T21:21:52.696217+0000","last_clean":"2024-09-15T21:21:52.696217+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:52.696217+0000","last_undegraded":"2024-09-15T21:21:52.696217+0000","last_fullsized":"2024-09-15T21:21:52.696217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9175","reported_seq":9305,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.884119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:52.884119+0000","last_peered":"2024-09-15T21:21:52.884119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:52.884119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9175,"log_dups_size":0,"ondisk_log_size":9175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37560320,"num_objects":9170,"num_object_clones":0,"num_object_copies":18340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":26400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5799},{"shards":"0,2","objects":3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9433","reported_seq":9554,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.797249+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:52.797249+0000","last_peered":"2024-09-15T21:21:52.797249+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:52.797249+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38612992,"num_objects":9427,"num_object_clones":0,"num_object_copies":18854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9183","reported_seq":9303,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:52.829739+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:52.829739+0000","last_peered":"2024-09-15T21:21:52.829739+0000","last_clean":"2024-09-15T21:21:52.829739+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:52.829739+0000","last_undegraded":"2024-09-15T21:21:52.829739+0000","last_fullsized":"2024-09-15T21:21:52.829739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9183,"log_dups_size":0,"ondisk_log_size":9183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605376,"num_objects":9181,"num_object_clones":0,"num_object_copies":18362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9204","reported_seq":9328,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.144412+0000","last_change":"2024-09-15T21:18:43.798229+0000","last_active":"2024-09-15T21:21:54.144412+0000","last_peered":"2024-09-15T21:21:54.144412+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:54.144412+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9204,"log_dups_size":0,"ondisk_log_size":9204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564416,"num_objects":9171,"num_object_clones":0,"num_object_copies":18342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7790,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3325},{"shards":"2","objects":5833}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9007","reported_seq":9130,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.771811+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:52.771811+0000","last_peered":"2024-09-15T21:21:52.771811+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:52.771811+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9007,"log_dups_size":0,"ondisk_log_size":9007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36859904,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6533,"num_write_kb":26132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9162","reported_seq":9272,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.076300+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:54.076300+0000","last_peered":"2024-09-15T21:21:54.076300+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:54.076300+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9162,"log_dups_size":0,"ondisk_log_size":9162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3249},{"shards":"2","objects":5857}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9295","reported_seq":9410,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.170778+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:54.170778+0000","last_peered":"2024-09-15T21:21:54.170778+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:54.170778+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9295,"log_dups_size":0,"ondisk_log_size":9295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37900288,"num_objects":9253,"num_object_clones":0,"num_object_copies":18506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3332},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9332","reported_seq":9457,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.676812+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:52.676812+0000","last_peered":"2024-09-15T21:21:52.676812+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:52.676812+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9332,"log_dups_size":0,"ondisk_log_size":9332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6639,"num_write_kb":26556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9200","reported_seq":9330,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.866375+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:52.866375+0000","last_peered":"2024-09-15T21:21:52.866375+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:52.866375+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9200,"log_dups_size":0,"ondisk_log_size":9200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37650432,"num_objects":9192,"num_object_clones":0,"num_object_copies":18384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":27140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9171","reported_seq":9295,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:52.835820+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:52.835820+0000","last_peered":"2024-09-15T21:21:52.835820+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:52.835820+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37548032,"num_objects":9167,"num_object_clones":0,"num_object_copies":18334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6818,"num_write_kb":27272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5802},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9298","reported_seq":15581,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:52.906296+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:52.906296+0000","last_peered":"2024-09-15T21:21:52.906296+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:52.906296+0000","last_undegraded":"2024-09-15T21:21:52.906296+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38036073,"num_objects":9287,"num_object_clones":0,"num_object_copies":18574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":9287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6958,"num_write_kb":27832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3077,"num_bytes_recovered":12591104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8891},{"shards":"0,2","objects":396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605565545,"num_objects":147844,"num_object_clones":0,"num_object_copies":295688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33699,"num_objects_misplaced":396,"num_objects_unfound":0,"num_objects_dirty":147844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119509,"num_write_kb":477696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6470,"num_bytes_recovered":26484736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237811200,"data_stored":1237734069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148095,"ondisk_log_size":148095,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953528,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":644460,"kb_used_data":405916,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93727380,"statfs":{"total":96636764160,"available":95976837120,"internally_reserved":0,"allocated":415657984,"data_stored":415392139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149755,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803912,"kb_used_data":497400,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93567928,"statfs":{"total":96636764160,"available":95813558272,"internally_reserved":0,"allocated":509337600,"data_stored":509071755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,0,3,10,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559452,"kb_used_data":307020,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812388,"statfs":{"total":96636764160,"available":96063885312,"internally_reserved":0,"allocated":314388480,"data_stored":314060013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":508956672,"data_stored":508953193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314159104,"data_stored":314093003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414695424,"data_stored":414687873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:01.030 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:01.670 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:01.671 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:01.746 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-15T21:22:00.546048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598758036,"num_objects":146185,"num_object_clones":0,"num_object_copies":292370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121598,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6876,"num_bytes_recovered":27865107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150185,"ondisk_log_size":150185,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2013432,"kb_used_data":1203456,"kb_used_omap":12,"kb_used_meta":809907,"kb_avail":281102088,"statfs":{"total":289910292480,"available":287848538112,"internally_reserved":0,"allocated":1232338944,"data_stored":1231470643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":829344778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,5,7,0,0,12,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":884760,"num_objects":217,"num_object_clones":0,"num_object_copies":434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3280,"num_objects_misplaced":2617,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4135,"num_write_kb":8701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":2428928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.149888"},"pg_stats":[{"pgid":"2.f","version":"46'9374","reported_seq":9467,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.958442+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:57.958442+0000","last_peered":"2024-09-15T21:21:57.958442+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:57.958442+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9374,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5779},{"shards":"0,2","objects":3249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9479","reported_seq":9575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.241698+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:59.241698+0000","last_peered":"2024-09-15T21:21:59.241698+0000","last_clean":"2024-09-15T21:21:59.241698+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:59.241698+0000","last_undegraded":"2024-09-15T21:21:59.241698+0000","last_fullsized":"2024-09-15T21:21:59.241698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8205,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9524","reported_seq":16440,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.799417+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:57.799417+0000","last_peered":"2024-09-15T21:21:57.799417+0000","last_clean":"2024-09-15T21:21:57.799417+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:57.799417+0000","last_undegraded":"2024-09-15T21:21:57.799417+0000","last_fullsized":"2024-09-15T21:21:57.799417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277144,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8247,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9444","reported_seq":9513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.276603+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:59.276603+0000","last_peered":"2024-09-15T21:21:59.276603+0000","last_clean":"2024-09-15T21:21:59.276603+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:59.276603+0000","last_undegraded":"2024-09-15T21:21:59.276603+0000","last_fullsized":"2024-09-15T21:21:59.276603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9483","reported_seq":9602,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.812618+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:57.812618+0000","last_peered":"2024-09-15T21:21:57.812618+0000","last_clean":"2024-09-15T21:21:57.812618+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:57.812618+0000","last_undegraded":"2024-09-15T21:21:57.812618+0000","last_fullsized":"2024-09-15T21:21:57.812618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9316","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.914344+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:57.914344+0000","last_peered":"2024-09-15T21:21:57.914344+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:57.914344+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9316,"log_dups_size":0,"ondisk_log_size":9316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5699},{"shards":"0,2","objects":3248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9581","reported_seq":9718,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.893345+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:57.893345+0000","last_peered":"2024-09-15T21:21:57.893345+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:57.893345+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7217,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5859},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9301","reported_seq":9421,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.867896+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:57.867896+0000","last_peered":"2024-09-15T21:21:57.867896+0000","last_clean":"2024-09-15T21:21:57.867896+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:57.867896+0000","last_undegraded":"2024-09-15T21:21:57.867896+0000","last_fullsized":"2024-09-15T21:21:57.867896+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7086,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9330","reported_seq":9494,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.298007+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:21:59.298007+0000","last_peered":"2024-09-15T21:21:59.298007+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:59.298007+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9330,"log_dups_size":0,"ondisk_log_size":9330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":122,"num_objects_misplaced":3197,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":53248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3197},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9139","reported_seq":9274,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.920689+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:57.920689+0000","last_peered":"2024-09-15T21:21:57.920689+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:57.920689+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":3223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9281","reported_seq":9395,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.282996+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:59.282996+0000","last_peered":"2024-09-15T21:21:59.282996+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:59.282996+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7832,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3130},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9417","reported_seq":9544,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.248014+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:59.248014+0000","last_peered":"2024-09-15T21:21:59.248014+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:59.248014+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9417,"log_dups_size":0,"ondisk_log_size":9417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3210},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9461","reported_seq":9613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.944983+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:57.944983+0000","last_peered":"2024-09-15T21:21:57.944983+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:57.944983+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9461,"log_dups_size":0,"ondisk_log_size":9461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37883904,"num_objects":9249,"num_object_clones":0,"num_object_copies":18498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9320","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.845366+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:57.845366+0000","last_peered":"2024-09-15T21:21:57.845366+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:57.845366+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37240832,"num_objects":9092,"num_object_clones":0,"num_object_copies":18184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6905,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5716},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9298","reported_seq":9442,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.872941+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:57.872941+0000","last_peered":"2024-09-15T21:21:57.872941+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:57.872941+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191680,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6945,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9434","reported_seq":16365,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:57.898668+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:57.898668+0000","last_peered":"2024-09-15T21:21:57.898668+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:57.898668+0000","last_undegraded":"2024-09-15T21:21:57.898668+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9434,"log_dups_size":0,"ondisk_log_size":9434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37618281,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":72,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7094,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":13918208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9113},{"shards":"0,2","objects":72}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":598758017,"num_objects":146183,"num_object_clones":0,"num_object_copies":292366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121596,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6875,"num_bytes_recovered":27865088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1230233600,"data_stored":1230152397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150182,"ondisk_log_size":150182,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640660,"kb_used_data":402116,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93731180,"statfs":{"total":96636764160,"available":95980728320,"internally_reserved":0,"allocated":411766784,"data_stored":411496867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149756,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807624,"kb_used_data":492788,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93564216,"statfs":{"total":96636764160,"available":95809757184,"internally_reserved":0,"allocated":504614912,"data_stored":504349067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":322367153},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,0,0,12],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565148,"kb_used_data":308552,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93806692,"statfs":{"total":96636764160,"available":96058052608,"internally_reserved":0,"allocated":315957248,"data_stored":315624709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504090624,"data_stored":504087145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315559936,"data_stored":315489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410583040,"data_stored":410575489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:01.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:02.438 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:02.439 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:02.491 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-15T21:22:02.398805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598758036,"num_objects":146185,"num_object_clones":0,"num_object_copies":292370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121598,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6876,"num_bytes_recovered":27865107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150185,"ondisk_log_size":150185,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2013432,"kb_used_data":1203456,"kb_used_omap":12,"kb_used_meta":809907,"kb_avail":281102088,"statfs":{"total":289910292480,"available":287848538112,"internally_reserved":0,"allocated":1232338944,"data_stored":1231470643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":829344778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,5,7,0,0,12,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6807528,"num_objects":-1661,"num_object_clones":0,"num_object_copies":-3322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3280,"num_objects_misplaced":2873,"num_objects_unfound":0,"num_objects_dirty":-1661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2087,"num_write_kb":849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002172"},"pg_stats":[{"pgid":"2.f","version":"46'9374","reported_seq":9467,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.958442+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:57.958442+0000","last_peered":"2024-09-15T21:21:57.958442+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:57.958442+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9374,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5779},{"shards":"0,2","objects":3249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9479","reported_seq":9575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.241698+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:59.241698+0000","last_peered":"2024-09-15T21:21:59.241698+0000","last_clean":"2024-09-15T21:21:59.241698+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:59.241698+0000","last_undegraded":"2024-09-15T21:21:59.241698+0000","last_fullsized":"2024-09-15T21:21:59.241698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8205,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9524","reported_seq":16440,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.799417+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:57.799417+0000","last_peered":"2024-09-15T21:21:57.799417+0000","last_clean":"2024-09-15T21:21:57.799417+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:57.799417+0000","last_undegraded":"2024-09-15T21:21:57.799417+0000","last_fullsized":"2024-09-15T21:21:57.799417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277144,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8247,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9444","reported_seq":9513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.276603+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:59.276603+0000","last_peered":"2024-09-15T21:21:59.276603+0000","last_clean":"2024-09-15T21:21:59.276603+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:59.276603+0000","last_undegraded":"2024-09-15T21:21:59.276603+0000","last_fullsized":"2024-09-15T21:21:59.276603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9483","reported_seq":9602,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.812618+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:57.812618+0000","last_peered":"2024-09-15T21:21:57.812618+0000","last_clean":"2024-09-15T21:21:57.812618+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:57.812618+0000","last_undegraded":"2024-09-15T21:21:57.812618+0000","last_fullsized":"2024-09-15T21:21:57.812618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9316","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.914344+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:57.914344+0000","last_peered":"2024-09-15T21:21:57.914344+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:57.914344+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9316,"log_dups_size":0,"ondisk_log_size":9316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5699},{"shards":"0,2","objects":3248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9581","reported_seq":9718,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.893345+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:57.893345+0000","last_peered":"2024-09-15T21:21:57.893345+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:57.893345+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7217,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5859},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9301","reported_seq":9421,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.867896+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:57.867896+0000","last_peered":"2024-09-15T21:21:57.867896+0000","last_clean":"2024-09-15T21:21:57.867896+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:57.867896+0000","last_undegraded":"2024-09-15T21:21:57.867896+0000","last_fullsized":"2024-09-15T21:21:57.867896+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7086,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9330","reported_seq":9494,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.298007+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:21:59.298007+0000","last_peered":"2024-09-15T21:21:59.298007+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:59.298007+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9330,"log_dups_size":0,"ondisk_log_size":9330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":122,"num_objects_misplaced":3197,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":53248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3197},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9139","reported_seq":9274,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.920689+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:57.920689+0000","last_peered":"2024-09-15T21:21:57.920689+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:57.920689+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":3223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9281","reported_seq":9395,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.282996+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:59.282996+0000","last_peered":"2024-09-15T21:21:59.282996+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:59.282996+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7832,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3130},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9417","reported_seq":9544,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.248014+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:59.248014+0000","last_peered":"2024-09-15T21:21:59.248014+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:59.248014+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9417,"log_dups_size":0,"ondisk_log_size":9417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3210},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9461","reported_seq":9613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.944983+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:57.944983+0000","last_peered":"2024-09-15T21:21:57.944983+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:57.944983+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9461,"log_dups_size":0,"ondisk_log_size":9461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37883904,"num_objects":9249,"num_object_clones":0,"num_object_copies":18498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9320","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.845366+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:57.845366+0000","last_peered":"2024-09-15T21:21:57.845366+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:57.845366+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37240832,"num_objects":9092,"num_object_clones":0,"num_object_copies":18184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6905,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5716},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9298","reported_seq":9442,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.872941+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:57.872941+0000","last_peered":"2024-09-15T21:21:57.872941+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:57.872941+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191680,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6945,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9434","reported_seq":16365,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:57.898668+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:57.898668+0000","last_peered":"2024-09-15T21:21:57.898668+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:57.898668+0000","last_undegraded":"2024-09-15T21:21:57.898668+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9434,"log_dups_size":0,"ondisk_log_size":9434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37618281,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":72,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7094,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":13918208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9113},{"shards":"0,2","objects":72}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":598758017,"num_objects":146183,"num_object_clones":0,"num_object_copies":292366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121596,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6875,"num_bytes_recovered":27865088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1230233600,"data_stored":1230152397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150182,"ondisk_log_size":150182,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640660,"kb_used_data":402116,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93731180,"statfs":{"total":96636764160,"available":95980728320,"internally_reserved":0,"allocated":411766784,"data_stored":411496867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149756,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807624,"kb_used_data":492788,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93564216,"statfs":{"total":96636764160,"available":95809757184,"internally_reserved":0,"allocated":504614912,"data_stored":504349067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":322367153},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,0,0,12],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565148,"kb_used_data":308552,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93806692,"statfs":{"total":96636764160,"available":96058052608,"internally_reserved":0,"allocated":315957248,"data_stored":315624709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504090624,"data_stored":504087145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315559936,"data_stored":315489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410583040,"data_stored":410575489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "46'9374", 'reported_seq': 9467, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.958442+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:21:57.958442+0000', 'last_peered': '2024-09-15T21:21:57.958442+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:21:57.958442+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9374, 'log_dups_size': 0, 'ondisk_log_size': 9374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37429248, 'num_objects': 9138, 'num_object_clones': 0, 'num_object_copies': 18276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3359, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9374, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5779}, {'shards': '0,2', 'objects': 3249}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "46'9316", 'reported_seq': 9456, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.914344+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:21:57.914344+0000', 'last_peered': '2024-09-15T21:21:57.914344+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:21:57.914344+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9316, 'log_dups_size': 0, 'ondisk_log_size': 9316, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37130240, 'num_objects': 9065, 'num_object_clones': 0, 'num_object_copies': 18130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3368, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6741, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5699}, {'shards': '0,2', 'objects': 3248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:02.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "46'9581", 'reported_seq': 9718, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.893345+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:21:57.893345+0000', 'last_peered': '2024-09-15T21:21:57.893345+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:21:57.893345+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9581, 'log_dups_size': 0, 'ondisk_log_size': 9581, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38187008, 'num_objects': 9323, 'num_object_clones': 0, 'num_object_copies': 18646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3466, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7217, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5859}, {'shards': '0,2', 'objects': 3343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "46'9330", 'reported_seq': 9494, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:59.298007+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:21:59.298007+0000', 'last_peered': '2024-09-15T21:21:59.298007+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:21:59.298007+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9330, 'log_dups_size': 0, 'ondisk_log_size': 9330, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37048320, 'num_objects': 9045, 'num_object_clones': 0, 'num_object_copies': 18090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 122, 'num_objects_misplaced': 3197, 'num_objects_unfound': 0, 'num_objects_dirty': 9045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7916, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 53248, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3197}, {'shards': '2', 'objects': 5726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "46'9139", 'reported_seq': 9274, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.920689+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:21:57.920689+0000', 'last_peered': '2024-09-15T21:21:57.920689+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:21:57.920689+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9139, 'log_dups_size': 0, 'ondisk_log_size': 9139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36433920, 'num_objects': 8895, 'num_object_clones': 0, 'num_object_copies': 17790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3337, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6665, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5560}, {'shards': '0,2', 'objects': 3223}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "46'9281", 'reported_seq': 9395, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:59.282996+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:21:59.282996+0000', 'last_peered': '2024-09-15T21:21:59.282996+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:21:59.282996+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9281, 'log_dups_size': 0, 'ondisk_log_size': 9281, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36880384, 'num_objects': 9004, 'num_object_clones': 0, 'num_object_copies': 18008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3264, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7832, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3130}, {'shards': '2', 'objects': 5742}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "46'9417", 'reported_seq': 9544, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:59.248014+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:21:59.248014+0000', 'last_peered': '2024-09-15T21:21:59.248014+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:21:59.248014+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9417, 'log_dups_size': 0, 'ondisk_log_size': 9417, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37400576, 'num_objects': 9131, 'num_object_clones': 0, 'num_object_copies': 18262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3346, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7816, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3210}, {'shards': '2', 'objects': 5787}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:02.493 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "46'9461", 'reported_seq': 9613, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.944983+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:21:57.944983+0000', 'last_peered': '2024-09-15T21:21:57.944983+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:21:57.944983+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9461, 'log_dups_size': 0, 'ondisk_log_size': 9461, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37883904, 'num_objects': 9249, 'num_object_clones': 0, 'num_object_copies': 18498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6768, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5829}, {'shards': '0,2', 'objects': 3328}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "46'9320", 'reported_seq': 9456, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.845366+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:21:57.845366+0000', 'last_peered': '2024-09-15T21:21:57.845366+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:21:57.845366+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9320, 'log_dups_size': 0, 'ondisk_log_size': 9320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37240832, 'num_objects': 9092, 'num_object_clones': 0, 'num_object_copies': 18184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3379, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6905, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5716}, {'shards': '0,2', 'objects': 3269}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "46'9298", 'reported_seq': 9442, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:57.872941+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:21:57.872941+0000', 'last_peered': '2024-09-15T21:21:57.872941+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:21:57.872941+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9298, 'log_dups_size': 0, 'ondisk_log_size': 9298, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37191680, 'num_objects': 9080, 'num_object_clones': 0, 'num_object_copies': 18160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3357, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9080, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6945, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5725}, {'shards': '0,2', 'objects': 3258}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:02.494 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "46'9434", 'reported_seq': 16365, 'reported_epoch': 46, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:57.898668+0000', 'last_change': '2024-09-15T21:20:15.068969+0000', 'last_active': '2024-09-15T21:21:57.898668+0000', 'last_peered': '2024-09-15T21:21:57.898668+0000', 'last_clean': '2024-09-15T21:18:41.321065+0000', 'last_became_active': '2024-09-15T21:18:43.762930+0000', 'last_became_peered': '2024-09-15T21:18:43.762930+0000', 'last_unstale': '2024-09-15T21:21:57.898668+0000', 'last_undegraded': '2024-09-15T21:21:57.898668+0000', 'last_fullsized': '2024-09-15T21:18:43.370018+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2344", 'last_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:20.389703+0000', 'objects_scrubbed': 2329, 'log_size': 9434, 'log_dups_size': 0, 'ondisk_log_size': 9434, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 655, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37618281, 'num_objects': 9185, 'num_object_clones': 0, 'num_object_copies': 18370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 72, 'num_objects_unfound': 0, 'num_objects_dirty': 9185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7094, 'num_write_kb': 27900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3401, 'num_bytes_recovered': 13918208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 9113}, {'shards': '0,2', 'objects': 72}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:02.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:03.280 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:03.280 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30419/292370 objects degraded (10.404%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":336,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146185,"data_bytes":598758036,"bytes_used":2061754368,"bytes_avail":287848538112,"bytes_total":289910292480,"degraded_objects":30419,"degraded_total":292370,"degraded_ratio":0.10404282245100387,"misplaced_objects":3269,"misplaced_total":292370,"misplaced_ratio":0.011181037726168897,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":172497,"recovering_keys_per_sec":0,"num_objects_recovered":405,"num_bytes_recovered":1380352,"num_keys_recovered":0,"read_bytes_sec":127,"write_bytes_sec":108642,"read_op_per_sec":0,"write_op_per_sec":260},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:03.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:03.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:04.101 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:04.102 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:04.147 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-15T21:22:02.398805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598758036,"num_objects":146185,"num_object_clones":0,"num_object_copies":292370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121598,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6876,"num_bytes_recovered":27865107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150185,"ondisk_log_size":150185,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2013432,"kb_used_data":1203456,"kb_used_omap":12,"kb_used_meta":809907,"kb_avail":281102088,"statfs":{"total":289910292480,"available":287848538112,"internally_reserved":0,"allocated":1232338944,"data_stored":1231470643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":829344778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,5,7,0,0,12,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6807528,"num_objects":-1661,"num_object_clones":0,"num_object_copies":-3322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3280,"num_objects_misplaced":2873,"num_objects_unfound":0,"num_objects_dirty":-1661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2087,"num_write_kb":849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002172"},"pg_stats":[{"pgid":"2.f","version":"46'9374","reported_seq":9467,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.958442+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:21:57.958442+0000","last_peered":"2024-09-15T21:21:57.958442+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:21:57.958442+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9374,"log_dups_size":0,"ondisk_log_size":9374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37429248,"num_objects":9138,"num_object_clones":0,"num_object_copies":18276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9374,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5779},{"shards":"0,2","objects":3249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'9479","reported_seq":9575,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.241698+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:21:59.241698+0000","last_peered":"2024-09-15T21:21:59.241698+0000","last_clean":"2024-09-15T21:21:59.241698+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:21:59.241698+0000","last_undegraded":"2024-09-15T21:21:59.241698+0000","last_fullsized":"2024-09-15T21:21:59.241698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8205,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'9524","reported_seq":16440,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.799417+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:21:57.799417+0000","last_peered":"2024-09-15T21:21:57.799417+0000","last_clean":"2024-09-15T21:21:57.799417+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:21:57.799417+0000","last_undegraded":"2024-09-15T21:21:57.799417+0000","last_fullsized":"2024-09-15T21:21:57.799417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277144,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8247,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"46'9444","reported_seq":9513,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:59.276603+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:21:59.276603+0000","last_peered":"2024-09-15T21:21:59.276603+0000","last_clean":"2024-09-15T21:21:59.276603+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:21:59.276603+0000","last_undegraded":"2024-09-15T21:21:59.276603+0000","last_fullsized":"2024-09-15T21:21:59.276603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9444,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'9483","reported_seq":9602,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.812618+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:21:57.812618+0000","last_peered":"2024-09-15T21:21:57.812618+0000","last_clean":"2024-09-15T21:21:57.812618+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:21:57.812618+0000","last_undegraded":"2024-09-15T21:21:57.812618+0000","last_fullsized":"2024-09-15T21:21:57.812618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9483,"log_dups_size":0,"ondisk_log_size":9483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7341,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'9316","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.914344+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:21:57.914344+0000","last_peered":"2024-09-15T21:21:57.914344+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:21:57.914344+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9316,"log_dups_size":0,"ondisk_log_size":9316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5699},{"shards":"0,2","objects":3248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'9581","reported_seq":9718,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.893345+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:21:57.893345+0000","last_peered":"2024-09-15T21:21:57.893345+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:21:57.893345+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7217,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5859},{"shards":"0,2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'9301","reported_seq":9421,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:21:57.867896+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:21:57.867896+0000","last_peered":"2024-09-15T21:21:57.867896+0000","last_clean":"2024-09-15T21:21:57.867896+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:21:57.867896+0000","last_undegraded":"2024-09-15T21:21:57.867896+0000","last_fullsized":"2024-09-15T21:21:57.867896+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9301,"log_dups_size":0,"ondisk_log_size":9301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7086,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.427444+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:20:16.427444+0000","last_peered":"2024-09-15T21:20:16.427444+0000","last_clean":"2024-09-15T21:20:16.427444+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:20:16.427444+0000","last_undegraded":"2024-09-15T21:20:16.427444+0000","last_fullsized":"2024-09-15T21:20:16.427444+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'9330","reported_seq":9494,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.298007+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:21:59.298007+0000","last_peered":"2024-09-15T21:21:59.298007+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:21:59.298007+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9330,"log_dups_size":0,"ondisk_log_size":9330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":122,"num_objects_misplaced":3197,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":53248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3197},{"shards":"2","objects":5726}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.436883+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:20:16.436883+0000","last_peered":"2024-09-15T21:20:16.436883+0000","last_clean":"2024-09-15T21:20:16.436883+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:20:16.436883+0000","last_undegraded":"2024-09-15T21:20:16.436883+0000","last_fullsized":"2024-09-15T21:20:16.436883+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'9139","reported_seq":9274,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.920689+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:21:57.920689+0000","last_peered":"2024-09-15T21:21:57.920689+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:21:57.920689+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3337,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":3223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":101,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.461374+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:20:16.461374+0000","last_peered":"2024-09-15T21:20:16.461374+0000","last_clean":"2024-09-15T21:20:16.461374+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:20:16.461374+0000","last_undegraded":"2024-09-15T21:20:16.461374+0000","last_fullsized":"2024-09-15T21:20:16.461374+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'9281","reported_seq":9395,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.282996+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:21:59.282996+0000","last_peered":"2024-09-15T21:21:59.282996+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:21:59.282996+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7832,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3130},{"shards":"2","objects":5742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430054+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:20:16.430054+0000","last_peered":"2024-09-15T21:20:16.430054+0000","last_clean":"2024-09-15T21:20:16.430054+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:20:16.430054+0000","last_undegraded":"2024-09-15T21:20:16.430054+0000","last_fullsized":"2024-09-15T21:20:16.430054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'9417","reported_seq":9544,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:59.248014+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:21:59.248014+0000","last_peered":"2024-09-15T21:21:59.248014+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:21:59.248014+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9417,"log_dups_size":0,"ondisk_log_size":9417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3210},{"shards":"2","objects":5787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"46'9461","reported_seq":9613,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.944983+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:21:57.944983+0000","last_peered":"2024-09-15T21:21:57.944983+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:21:57.944983+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9461,"log_dups_size":0,"ondisk_log_size":9461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37883904,"num_objects":9249,"num_object_clones":0,"num_object_copies":18498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.431456+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:20:16.431456+0000","last_peered":"2024-09-15T21:20:16.431456+0000","last_clean":"2024-09-15T21:20:16.431456+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:20:16.431456+0000","last_undegraded":"2024-09-15T21:20:16.431456+0000","last_fullsized":"2024-09-15T21:20:16.431456+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"46'9320","reported_seq":9456,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.845366+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:21:57.845366+0000","last_peered":"2024-09-15T21:21:57.845366+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:21:57.845366+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37240832,"num_objects":9092,"num_object_clones":0,"num_object_copies":18184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6905,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5716},{"shards":"0,2","objects":3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.437737+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:20:16.437737+0000","last_peered":"2024-09-15T21:20:16.437737+0000","last_clean":"2024-09-15T21:20:16.437737+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:20:16.437737+0000","last_undegraded":"2024-09-15T21:20:16.437737+0000","last_fullsized":"2024-09-15T21:20:16.437737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'9298","reported_seq":9442,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:57.872941+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:21:57.872941+0000","last_peered":"2024-09-15T21:21:57.872941+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:21:57.872941+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37191680,"num_objects":9080,"num_object_clones":0,"num_object_copies":18160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6945,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":3258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430832+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:20:16.430832+0000","last_peered":"2024-09-15T21:20:16.430832+0000","last_clean":"2024-09-15T21:20:16.430832+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:20:16.430832+0000","last_undegraded":"2024-09-15T21:20:16.430832+0000","last_fullsized":"2024-09-15T21:20:16.430832+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:20:16.430146+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:20:16.430146+0000","last_peered":"2024-09-15T21:20:16.430146+0000","last_clean":"2024-09-15T21:20:16.430146+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:20:16.430146+0000","last_undegraded":"2024-09-15T21:20:16.430146+0000","last_fullsized":"2024-09-15T21:20:16.430146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'9434","reported_seq":16365,"reported_epoch":46,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:57.898668+0000","last_change":"2024-09-15T21:20:15.068969+0000","last_active":"2024-09-15T21:21:57.898668+0000","last_peered":"2024-09-15T21:21:57.898668+0000","last_clean":"2024-09-15T21:18:41.321065+0000","last_became_active":"2024-09-15T21:18:43.762930+0000","last_became_peered":"2024-09-15T21:18:43.762930+0000","last_unstale":"2024-09-15T21:21:57.898668+0000","last_undegraded":"2024-09-15T21:21:57.898668+0000","last_fullsized":"2024-09-15T21:18:43.370018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9434,"log_dups_size":0,"ondisk_log_size":9434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37618281,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":72,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7094,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":13918208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":9113},{"shards":"0,2","objects":72}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":598758017,"num_objects":146183,"num_object_clones":0,"num_object_copies":292366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30419,"num_objects_misplaced":3269,"num_objects_unfound":0,"num_objects_dirty":146183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":121596,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6875,"num_bytes_recovered":27865088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1230233600,"data_stored":1230152397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150182,"ondisk_log_size":150182,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953529,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640660,"kb_used_data":402116,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93731180,"statfs":{"total":96636764160,"available":95980728320,"internally_reserved":0,"allocated":411766784,"data_stored":411496867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,0,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149756,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807624,"kb_used_data":492788,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93564216,"statfs":{"total":96636764160,"available":95809757184,"internally_reserved":0,"allocated":504614912,"data_stored":504349067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":322367153},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,2,0,0,12],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565148,"kb_used_data":308552,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93806692,"statfs":{"total":96636764160,"available":96058052608,"internally_reserved":0,"allocated":315957248,"data_stored":315624709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":504090624,"data_stored":504087145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315559936,"data_stored":315489763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410583040,"data_stored":410575489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:07.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:07.884 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:07.939 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-15T21:22:06.399821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590951060,"num_objects":144279,"num_object_clones":0,"num_object_copies":288558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30387,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":144279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123504,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7136,"num_bytes_recovered":28794899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152091,"ondisk_log_size":152091,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1970124,"kb_used_data":1193432,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281145396,"statfs":{"total":289910292480,"available":287892885504,"internally_reserved":0,"allocated":1222074368,"data_stored":1221174732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":795266058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,2,5,0,0,5,9,2,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10448896,"num_objects":-2551,"num_object_clones":0,"num_object_copies":-5102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3260,"num_objects_misplaced":2969,"num_objects_unfound":0,"num_objects_dirty":-2551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2551,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002329"},"pg_stats":[{"pgid":"2.f","version":"48'9490","reported_seq":9596,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.907220+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:02.907220+0000","last_peered":"2024-09-15T21:22:02.907220+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:02.907220+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36954112,"num_objects":9022,"num_object_clones":0,"num_object_copies":18044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5668},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9601","reported_seq":9701,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.168340+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:04.168340+0000","last_peered":"2024-09-15T21:22:04.168340+0000","last_clean":"2024-09-15T21:22:04.168340+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:04.168340+0000","last_undegraded":"2024-09-15T21:22:04.168340+0000","last_fullsized":"2024-09-15T21:22:04.168340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8327,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9652","reported_seq":16572,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:02.824756+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:02.824756+0000","last_peered":"2024-09-15T21:22:02.824756+0000","last_clean":"2024-09-15T21:22:02.824756+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:02.824756+0000","last_undegraded":"2024-09-15T21:22:02.824756+0000","last_fullsized":"2024-09-15T21:22:02.824756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9652,"log_dups_size":0,"ondisk_log_size":9652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37752856,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8375,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9552","reported_seq":9625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.249003+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:04.249003+0000","last_peered":"2024-09-15T21:22:04.249003+0000","last_clean":"2024-09-15T21:22:04.249003+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:04.249003+0000","last_undegraded":"2024-09-15T21:22:04.249003+0000","last_fullsized":"2024-09-15T21:22:04.249003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9584","reported_seq":9707,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:02.901407+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:02.901407+0000","last_peered":"2024-09-15T21:22:02.901407+0000","last_clean":"2024-09-15T21:22:02.901407+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:02.901407+0000","last_undegraded":"2024-09-15T21:22:02.901407+0000","last_fullsized":"2024-09-15T21:22:02.901407+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37433344,"num_objects":9139,"num_object_clones":0,"num_object_copies":18278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9441","reported_seq":9595,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.874259+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:02.874259+0000","last_peered":"2024-09-15T21:22:02.874259+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:02.874259+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9441,"log_dups_size":0,"ondisk_log_size":9441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618240,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5579},{"shards":"0,2","objects":3123}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9704","reported_seq":9847,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.942708+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:02.942708+0000","last_peered":"2024-09-15T21:22:02.942708+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:02.942708+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9704,"log_dups_size":0,"ondisk_log_size":9704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5737},{"shards":"0,2","objects":3220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9424","reported_seq":9548,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:02.911480+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:02.911480+0000","last_peered":"2024-09-15T21:22:02.911480+0000","last_clean":"2024-09-15T21:22:02.911480+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:02.911480+0000","last_undegraded":"2024-09-15T21:22:02.911480+0000","last_fullsized":"2024-09-15T21:22:02.911480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9424,"log_dups_size":0,"ondisk_log_size":9424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7209,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9445","reported_seq":9923,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.298658+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:04.298658+0000","last_peered":"2024-09-15T21:22:04.298658+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:04.298658+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9445,"log_dups_size":0,"ondisk_log_size":9445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8031,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3041},{"shards":"2","objects":5766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9257","reported_seq":9398,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.926442+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:02.926442+0000","last_peered":"2024-09-15T21:22:02.926442+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:02.926442+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9257,"log_dups_size":0,"ondisk_log_size":9257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35950592,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6783,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":3105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9392","reported_seq":9524,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.236878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:04.236878+0000","last_peered":"2024-09-15T21:22:04.236878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:04.236878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7943,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3019},{"shards":"2","objects":5636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9520","reported_seq":9653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.268725+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:04.268725+0000","last_peered":"2024-09-15T21:22:04.268725+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:04.268725+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36978688,"num_objects":9028,"num_object_clones":0,"num_object_copies":18056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3107},{"shards":"2","objects":5685}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9588","reported_seq":9754,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.001267+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:03.001267+0000","last_peered":"2024-09-15T21:22:03.001267+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:03.001267+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9588,"log_dups_size":0,"ondisk_log_size":9588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5706},{"shards":"0,2","objects":3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9458","reported_seq":9607,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.922261+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:02.922261+0000","last_peered":"2024-09-15T21:22:02.922261+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:02.922261+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9458,"log_dups_size":0,"ondisk_log_size":9458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7043,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5583},{"shards":"0,2","objects":3131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9428","reported_seq":9588,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:02.858080+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:02.858080+0000","last_peered":"2024-09-15T21:22:02.858080+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:02.858080+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36659200,"num_objects":8950,"num_object_clones":0,"num_object_copies":17900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7075,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5601},{"shards":"0,2","objects":3128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9552","reported_seq":16641,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:02.780204+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:02.780204+0000","last_peered":"2024-09-15T21:22:02.780204+0000","last_clean":"2024-09-15T21:22:02.780204+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:02.780204+0000","last_undegraded":"2024-09-15T21:22:02.780204+0000","last_fullsized":"2024-09-15T21:22:02.780204+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134953,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7212,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":590951041,"num_objects":144277,"num_object_clones":0,"num_object_copies":288554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30387,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":144277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123502,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7135,"num_bytes_recovered":28794880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1219952640,"data_stored":1219871437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152088,"ondisk_log_size":152088,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624360,"kb_used_data":398292,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93747480,"statfs":{"total":96636764160,"available":95997419520,"internally_reserved":0,"allocated":407851008,"data_stored":407570646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,0,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149757,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789436,"kb_used_data":487084,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93582404,"statfs":{"total":96636764160,"available":95828381696,"internally_reserved":0,"allocated":498774016,"data_stored":498497726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,2,5,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626475,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556328,"kb_used_data":308056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93815512,"statfs":{"total":96636764160,"available":96067084288,"internally_reserved":0,"allocated":315449344,"data_stored":315106360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,5,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498106368,"data_stored":498102889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315039744,"data_stored":314969571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406806528,"data_stored":406798977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:07.941 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:08.534 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:08.588 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-15T21:22:08.400153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585220756,"num_objects":142880,"num_object_clones":0,"num_object_copies":285760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30299,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":142880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":124903,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7224,"num_bytes_recovered":28794899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153490,"ondisk_log_size":153490,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1971504,"kb_used_data":1186480,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281144016,"statfs":{"total":289910292480,"available":287891472384,"internally_reserved":0,"allocated":1214955520,"data_stored":1214055884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":803785738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,5,3,0,7,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13537280,"num_objects":-3305,"num_object_clones":0,"num_object_copies":-6610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":-3305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":929792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.856127"},"pg_stats":[{"pgid":"2.f","version":"48'9607","reported_seq":9740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.846545+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:07.846545+0000","last_peered":"2024-09-15T21:22:07.846545+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:07.846545+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9607,"log_dups_size":0,"ondisk_log_size":9607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3016}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9601","reported_seq":9701,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.168340+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:04.168340+0000","last_peered":"2024-09-15T21:22:04.168340+0000","last_clean":"2024-09-15T21:22:04.168340+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:04.168340+0000","last_undegraded":"2024-09-15T21:22:04.168340+0000","last_fullsized":"2024-09-15T21:22:04.168340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8327,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9800","reported_seq":16720,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.995780+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:07.995780+0000","last_peered":"2024-09-15T21:22:07.995780+0000","last_clean":"2024-09-15T21:22:07.995780+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:07.995780+0000","last_undegraded":"2024-09-15T21:22:07.995780+0000","last_fullsized":"2024-09-15T21:22:07.995780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9800,"log_dups_size":0,"ondisk_log_size":9800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146648,"num_objects":9070,"num_object_clones":0,"num_object_copies":18140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9552","reported_seq":9625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.249003+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:04.249003+0000","last_peered":"2024-09-15T21:22:04.249003+0000","last_clean":"2024-09-15T21:22:04.249003+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:04.249003+0000","last_undegraded":"2024-09-15T21:22:04.249003+0000","last_fullsized":"2024-09-15T21:22:04.249003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9706","reported_seq":9829,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.014983+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:08.014983+0000","last_peered":"2024-09-15T21:22:08.014983+0000","last_clean":"2024-09-15T21:22:08.014983+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:08.014983+0000","last_undegraded":"2024-09-15T21:22:08.014983+0000","last_fullsized":"2024-09-15T21:22:08.014983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9559","reported_seq":9741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.928417+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:07.928417+0000","last_peered":"2024-09-15T21:22:07.928417+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:07.928417+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9559,"log_dups_size":0,"ondisk_log_size":9559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9820","reported_seq":9987,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.831213+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:07.831213+0000","last_peered":"2024-09-15T21:22:07.831213+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:07.831213+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9548","reported_seq":9672,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.912283+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:07.912283+0000","last_peered":"2024-09-15T21:22:07.912283+0000","last_clean":"2024-09-15T21:22:07.912283+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:07.912283+0000","last_undegraded":"2024-09-15T21:22:07.912283+0000","last_fullsized":"2024-09-15T21:22:07.912283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7333,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9445","reported_seq":9923,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.298658+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:04.298658+0000","last_peered":"2024-09-15T21:22:04.298658+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:04.298658+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9445,"log_dups_size":0,"ondisk_log_size":9445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8031,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3041},{"shards":"2","objects":5766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9396","reported_seq":9562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.975232+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:07.975232+0000","last_peered":"2024-09-15T21:22:07.975232+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:07.975232+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9396,"log_dups_size":0,"ondisk_log_size":9396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6922,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9392","reported_seq":9524,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.236878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:04.236878+0000","last_peered":"2024-09-15T21:22:04.236878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:04.236878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7943,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3019},{"shards":"2","objects":5636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9520","reported_seq":9653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.268725+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:04.268725+0000","last_peered":"2024-09-15T21:22:04.268725+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:04.268725+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36978688,"num_objects":9028,"num_object_clones":0,"num_object_copies":18056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3107},{"shards":"2","objects":5685}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9728","reported_seq":9922,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.999651+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:07.999651+0000","last_peered":"2024-09-15T21:22:07.999651+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:07.999651+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9728,"log_dups_size":0,"ondisk_log_size":9728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36790272,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9584","reported_seq":9752,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.973384+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:07.973384+0000","last_peered":"2024-09-15T21:22:07.973384+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:07.973384+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7169,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9546","reported_seq":9732,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.978787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:07.978787+0000","last_peered":"2024-09-15T21:22:07.978787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:07.978787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5496},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9683","reported_seq":16772,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.944243+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:07.944243+0000","last_peered":"2024-09-15T21:22:07.944243+0000","last_clean":"2024-09-15T21:22:07.944243+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:07.944243+0000","last_undegraded":"2024-09-15T21:22:07.944243+0000","last_fullsized":"2024-09-15T21:22:07.944243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36598377,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585220737,"num_objects":142878,"num_object_clones":0,"num_object_copies":285756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30299,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":142878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":124901,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7223,"num_bytes_recovered":28794880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212760064,"data_stored":1212678861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153487,"ondisk_log_size":153487,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624360,"kb_used_data":398292,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93747480,"statfs":{"total":96636764160,"available":95997419520,"internally_reserved":0,"allocated":407851008,"data_stored":407570646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,0,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787596,"kb_used_data":481072,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93584244,"statfs":{"total":96636764160,"available":95830265856,"internally_reserved":0,"allocated":492617728,"data_stored":492341438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559548,"kb_used_data":307116,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812292,"statfs":{"total":96636764160,"available":96063787008,"internally_reserved":0,"allocated":314486784,"data_stored":314143800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491921408,"data_stored":491917929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314032128,"data_stored":313961955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406806528,"data_stored":406798977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:08.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:08.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'9607", 'reported_seq': 9740, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.846545+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:07.846545+0000', 'last_peered': '2024-09-15T21:22:07.846545+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:07.846545+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9607, 'log_dups_size': 0, 'ondisk_log_size': 9607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36474880, 'num_objects': 8905, 'num_object_clones': 0, 'num_object_copies': 17810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9607, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5564}, {'shards': '0,2', 'objects': 3016}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:08.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'9559", 'reported_seq': 9741, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.928417+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:07.928417+0000', 'last_peered': '2024-09-15T21:22:07.928417+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:07.928417+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9559, 'log_dups_size': 0, 'ondisk_log_size': 9559, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36134912, 'num_objects': 8822, 'num_object_clones': 0, 'num_object_copies': 17644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3349, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6984, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5475}, {'shards': '0,2', 'objects': 3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'9820", 'reported_seq': 9987, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.831213+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:07.831213+0000', 'last_peered': '2024-09-15T21:22:07.831213+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:07.831213+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9820, 'log_dups_size': 0, 'ondisk_log_size': 9820, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37208064, 'num_objects': 9084, 'num_object_clones': 0, 'num_object_copies': 18168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7456, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5633}, {'shards': '0,2', 'objects': 3104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'9445", 'reported_seq': 9923, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:04.298658+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:04.298658+0000', 'last_peered': '2024-09-15T21:22:04.298658+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:04.298658+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9445, 'log_dups_size': 0, 'ondisk_log_size': 9445, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36577280, 'num_objects': 8930, 'num_object_clones': 0, 'num_object_copies': 17860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 3041, 'num_objects_unfound': 0, 'num_objects_dirty': 8930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8031, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 174, 'num_bytes_recovered': 688128, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3041}, {'shards': '2', 'objects': 5766}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'9396", 'reported_seq': 9562, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.975232+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:07.975232+0000', 'last_peered': '2024-09-15T21:22:07.975232+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:07.975232+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9396, 'log_dups_size': 0, 'ondisk_log_size': 9396, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35381248, 'num_objects': 8638, 'num_object_clones': 0, 'num_object_copies': 17276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6922, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5317}, {'shards': '0,2', 'objects': 2966}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'9392", 'reported_seq': 9524, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:04.236878+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:04.236878+0000', 'last_peered': '2024-09-15T21:22:04.236878+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:04.236878+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9392, 'log_dups_size': 0, 'ondisk_log_size': 9392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36425728, 'num_objects': 8893, 'num_object_clones': 0, 'num_object_copies': 17786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7943, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 9, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3019}, {'shards': '2', 'objects': 5636}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'9520", 'reported_seq': 9653, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:04.268725+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:04.268725+0000', 'last_peered': '2024-09-15T21:22:04.268725+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:04.268725+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9520, 'log_dups_size': 0, 'ondisk_log_size': 9520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36978688, 'num_objects': 9028, 'num_object_clones': 0, 'num_object_copies': 18056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3345, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9028, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7919, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3107}, {'shards': '2', 'objects': 5685}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:08.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'9728", 'reported_seq': 9922, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.999651+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:07.999651+0000', 'last_peered': '2024-09-15T21:22:07.999651+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:07.999651+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9728, 'log_dups_size': 0, 'ondisk_log_size': 9728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36790272, 'num_objects': 8982, 'num_object_clones': 0, 'num_object_copies': 17964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3403, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7035, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5580}, {'shards': '0,2', 'objects': 3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:08.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'9584", 'reported_seq': 9752, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.973384+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:07.973384+0000', 'last_peered': '2024-09-15T21:22:07.973384+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:07.973384+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9584, 'log_dups_size': 0, 'ondisk_log_size': 9584, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36159488, 'num_objects': 8828, 'num_object_clones': 0, 'num_object_copies': 17656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7169, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5466}, {'shards': '0,2', 'objects': 3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:08.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'9546", 'reported_seq': 9732, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:07.978787+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:07.978787+0000', 'last_peered': '2024-09-15T21:22:07.978787+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:07.978787+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9546, 'log_dups_size': 0, 'ondisk_log_size': 9546, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36175872, 'num_objects': 8832, 'num_object_clones': 0, 'num_object_copies': 17664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3338, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7193, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5496}, {'shards': '0,2', 'objects': 3010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:08.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:09.401 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:09.402 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30299/285760 objects degraded (10.603%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":342,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142880,"data_bytes":585220756,"bytes_used":2018820096,"bytes_avail":287891472384,"bytes_total":289910292480,"degraded_objects":30299,"degraded_total":285760,"degraded_ratio":0.10602953527435609,"misplaced_objects":3041,"misplaced_total":285760,"misplaced_ratio":0.01064179731243001,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":104988,"recovering_keys_per_sec":0,"num_objects_recovered":348,"num_bytes_recovered":929792,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":373},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:09.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:09.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:10.307 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:10.309 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:10.336 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-15T21:22:08.400153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585220756,"num_objects":142880,"num_object_clones":0,"num_object_copies":285760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30299,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":142880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":124903,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7224,"num_bytes_recovered":28794899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153490,"ondisk_log_size":153490,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1971504,"kb_used_data":1186480,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281144016,"statfs":{"total":289910292480,"available":287891472384,"internally_reserved":0,"allocated":1214955520,"data_stored":1214055884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":803785738},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,5,3,0,7,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13537280,"num_objects":-3305,"num_object_clones":0,"num_object_copies":-6610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":-3305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":929792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.856127"},"pg_stats":[{"pgid":"2.f","version":"48'9607","reported_seq":9740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.846545+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:07.846545+0000","last_peered":"2024-09-15T21:22:07.846545+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:07.846545+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9607,"log_dups_size":0,"ondisk_log_size":9607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3016}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9601","reported_seq":9701,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.168340+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:04.168340+0000","last_peered":"2024-09-15T21:22:04.168340+0000","last_clean":"2024-09-15T21:22:04.168340+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:04.168340+0000","last_undegraded":"2024-09-15T21:22:04.168340+0000","last_fullsized":"2024-09-15T21:22:04.168340+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37023744,"num_objects":9039,"num_object_clones":0,"num_object_copies":18078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8327,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9800","reported_seq":16720,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.995780+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:07.995780+0000","last_peered":"2024-09-15T21:22:07.995780+0000","last_clean":"2024-09-15T21:22:07.995780+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:07.995780+0000","last_undegraded":"2024-09-15T21:22:07.995780+0000","last_fullsized":"2024-09-15T21:22:07.995780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9800,"log_dups_size":0,"ondisk_log_size":9800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146648,"num_objects":9070,"num_object_clones":0,"num_object_copies":18140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9552","reported_seq":9625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:04.249003+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:04.249003+0000","last_peered":"2024-09-15T21:22:04.249003+0000","last_clean":"2024-09-15T21:22:04.249003+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:04.249003+0000","last_undegraded":"2024-09-15T21:22:04.249003+0000","last_fullsized":"2024-09-15T21:22:04.249003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9552,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9706","reported_seq":9829,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.014983+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:08.014983+0000","last_peered":"2024-09-15T21:22:08.014983+0000","last_clean":"2024-09-15T21:22:08.014983+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:08.014983+0000","last_undegraded":"2024-09-15T21:22:08.014983+0000","last_fullsized":"2024-09-15T21:22:08.014983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9559","reported_seq":9741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.928417+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:07.928417+0000","last_peered":"2024-09-15T21:22:07.928417+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:07.928417+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9559,"log_dups_size":0,"ondisk_log_size":9559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9820","reported_seq":9987,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.831213+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:07.831213+0000","last_peered":"2024-09-15T21:22:07.831213+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:07.831213+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9548","reported_seq":9672,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.912283+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:07.912283+0000","last_peered":"2024-09-15T21:22:07.912283+0000","last_clean":"2024-09-15T21:22:07.912283+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:07.912283+0000","last_undegraded":"2024-09-15T21:22:07.912283+0000","last_fullsized":"2024-09-15T21:22:07.912283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7333,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9445","reported_seq":9923,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.298658+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:04.298658+0000","last_peered":"2024-09-15T21:22:04.298658+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:04.298658+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9445,"log_dups_size":0,"ondisk_log_size":9445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8031,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3041},{"shards":"2","objects":5766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9396","reported_seq":9562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.975232+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:07.975232+0000","last_peered":"2024-09-15T21:22:07.975232+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:07.975232+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9396,"log_dups_size":0,"ondisk_log_size":9396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6922,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9392","reported_seq":9524,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.236878+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:04.236878+0000","last_peered":"2024-09-15T21:22:04.236878+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:04.236878+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7943,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3019},{"shards":"2","objects":5636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9520","reported_seq":9653,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:04.268725+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:04.268725+0000","last_peered":"2024-09-15T21:22:04.268725+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:04.268725+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36978688,"num_objects":9028,"num_object_clones":0,"num_object_copies":18056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7919,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3107},{"shards":"2","objects":5685}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9728","reported_seq":9922,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.999651+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:07.999651+0000","last_peered":"2024-09-15T21:22:07.999651+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:07.999651+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9728,"log_dups_size":0,"ondisk_log_size":9728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36790272,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9584","reported_seq":9752,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.973384+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:07.973384+0000","last_peered":"2024-09-15T21:22:07.973384+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:07.973384+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7169,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9546","reported_seq":9732,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.978787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:07.978787+0000","last_peered":"2024-09-15T21:22:07.978787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:07.978787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5496},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9683","reported_seq":16772,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.944243+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:07.944243+0000","last_peered":"2024-09-15T21:22:07.944243+0000","last_clean":"2024-09-15T21:22:07.944243+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:07.944243+0000","last_undegraded":"2024-09-15T21:22:07.944243+0000","last_fullsized":"2024-09-15T21:22:07.944243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36598377,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585220737,"num_objects":142878,"num_object_clones":0,"num_object_copies":285756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30299,"num_objects_misplaced":3041,"num_objects_unfound":0,"num_objects_dirty":142878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":124901,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7223,"num_bytes_recovered":28794880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212760064,"data_stored":1212678861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153487,"ondisk_log_size":153487,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624360,"kb_used_data":398292,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93747480,"statfs":{"total":96636764160,"available":95997419520,"internally_reserved":0,"allocated":407851008,"data_stored":407570646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,0,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787596,"kb_used_data":481072,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93584244,"statfs":{"total":96636764160,"available":95830265856,"internally_reserved":0,"allocated":492617728,"data_stored":492341438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559548,"kb_used_data":307116,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812292,"statfs":{"total":96636764160,"available":96063787008,"internally_reserved":0,"allocated":314486784,"data_stored":314143800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491921408,"data_stored":491917929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314032128,"data_stored":313961955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406806528,"data_stored":406798977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:13.338 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:14.133 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:14.175 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-15T21:22:12.401054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":582648468,"num_objects":142252,"num_object_clones":0,"num_object_copies":284504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30283,"num_objects_misplaced":2893,"num_objects_unfound":0,"num_objects_dirty":142252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125531,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7388,"num_bytes_recovered":29401107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154118,"ondisk_log_size":154118,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1971608,"kb_used_data":1182424,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281143912,"statfs":{"total":289910292480,"available":287891365888,"internally_reserved":0,"allocated":1210802176,"data_stored":1209902540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":808045578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,6,6,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16109568,"num_objects":-3933,"num_object_clones":0,"num_object_copies":-7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-136,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":-3933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.855005"},"pg_stats":[{"pgid":"2.f","version":"48'9607","reported_seq":9740,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.846545+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:07.846545+0000","last_peered":"2024-09-15T21:22:07.846545+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:07.846545+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9607,"log_dups_size":0,"ondisk_log_size":9607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5564},{"shards":"0,2","objects":3016}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9726","reported_seq":9827,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:09.323931+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:09.323931+0000","last_peered":"2024-09-15T21:22:09.323931+0000","last_clean":"2024-09-15T21:22:09.323931+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:09.323931+0000","last_undegraded":"2024-09-15T21:22:09.323931+0000","last_fullsized":"2024-09-15T21:22:09.323931+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8452,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9800","reported_seq":16720,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.995780+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:07.995780+0000","last_peered":"2024-09-15T21:22:07.995780+0000","last_clean":"2024-09-15T21:22:07.995780+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:07.995780+0000","last_undegraded":"2024-09-15T21:22:07.995780+0000","last_fullsized":"2024-09-15T21:22:07.995780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9800,"log_dups_size":0,"ondisk_log_size":9800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146648,"num_objects":9070,"num_object_clones":0,"num_object_copies":18140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8523,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9681","reported_seq":9754,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:09.312072+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:09.312072+0000","last_peered":"2024-09-15T21:22:09.312072+0000","last_clean":"2024-09-15T21:22:09.312072+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:09.312072+0000","last_undegraded":"2024-09-15T21:22:09.312072+0000","last_fullsized":"2024-09-15T21:22:09.312072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9681,"log_dups_size":0,"ondisk_log_size":9681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9681,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9706","reported_seq":9829,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.014983+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:08.014983+0000","last_peered":"2024-09-15T21:22:08.014983+0000","last_clean":"2024-09-15T21:22:08.014983+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:08.014983+0000","last_undegraded":"2024-09-15T21:22:08.014983+0000","last_fullsized":"2024-09-15T21:22:08.014983+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36933632,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9559","reported_seq":9741,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.928417+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:07.928417+0000","last_peered":"2024-09-15T21:22:07.928417+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:07.928417+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9559,"log_dups_size":0,"ondisk_log_size":9559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9820","reported_seq":9987,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.831213+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:07.831213+0000","last_peered":"2024-09-15T21:22:07.831213+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:07.831213+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9820,"log_dups_size":0,"ondisk_log_size":9820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9548","reported_seq":9672,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.912283+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:07.912283+0000","last_peered":"2024-09-15T21:22:07.912283+0000","last_clean":"2024-09-15T21:22:07.912283+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:07.912283+0000","last_undegraded":"2024-09-15T21:22:07.912283+0000","last_fullsized":"2024-09-15T21:22:07.912283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7333,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9555","reported_seq":10329,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:09.299226+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:09.299226+0000","last_peered":"2024-09-15T21:22:09.299226+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:09.299226+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36126720,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2893,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8141,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2893},{"shards":"2","objects":5804}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9396","reported_seq":9562,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.975232+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:07.975232+0000","last_peered":"2024-09-15T21:22:07.975232+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:07.975232+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9396,"log_dups_size":0,"ondisk_log_size":9396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6922,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9526","reported_seq":9678,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:09.232809+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:09.232809+0000","last_peered":"2024-09-15T21:22:09.232809+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:09.232809+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9526,"log_dups_size":0,"ondisk_log_size":9526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2885},{"shards":"2","objects":5512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9650","reported_seq":9795,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:09.203946+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:09.203946+0000","last_peered":"2024-09-15T21:22:09.203946+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:09.203946+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9650,"log_dups_size":0,"ondisk_log_size":9650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8049,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2977},{"shards":"2","objects":5561}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9728","reported_seq":9922,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.999651+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:07.999651+0000","last_peered":"2024-09-15T21:22:07.999651+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:07.999651+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9728,"log_dups_size":0,"ondisk_log_size":9728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36790272,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7035,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9584","reported_seq":9752,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.973384+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:07.973384+0000","last_peered":"2024-09-15T21:22:07.973384+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:07.973384+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9584,"log_dups_size":0,"ondisk_log_size":9584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36159488,"num_objects":8828,"num_object_clones":0,"num_object_copies":17656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7169,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9546","reported_seq":9732,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:07.978787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:07.978787+0000","last_peered":"2024-09-15T21:22:07.978787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:07.978787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5496},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9683","reported_seq":16772,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:07.944243+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:07.944243+0000","last_peered":"2024-09-15T21:22:07.944243+0000","last_clean":"2024-09-15T21:22:07.944243+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:07.944243+0000","last_undegraded":"2024-09-15T21:22:07.944243+0000","last_fullsized":"2024-09-15T21:22:07.944243+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9683,"log_dups_size":0,"ondisk_log_size":9683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36598377,"num_objects":8936,"num_object_clones":0,"num_object_copies":17872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":582648449,"num_objects":142250,"num_object_clones":0,"num_object_copies":284500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30283,"num_objects_misplaced":2893,"num_objects_unfound":0,"num_objects_dirty":142250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125529,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7387,"num_bytes_recovered":29401088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208655872,"data_stored":1208574669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154115,"ondisk_log_size":154115,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953531,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624464,"kb_used_data":394236,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93747376,"statfs":{"total":96636764160,"available":95997313024,"internally_reserved":0,"allocated":403697664,"data_stored":403417302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787596,"kb_used_data":481072,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93584244,"statfs":{"total":96636764160,"available":95830265856,"internally_reserved":0,"allocated":492617728,"data_stored":492341438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,3,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559548,"kb_used_data":307116,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812292,"statfs":{"total":96636764160,"available":96063787008,"internally_reserved":0,"allocated":314486784,"data_stored":314143800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":258469548},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491921408,"data_stored":491917929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314032128,"data_stored":313961955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402702336,"data_stored":402694785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:14.176 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:14.870 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:14.928 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-15T21:22:14.401515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":573952660,"num_objects":140129,"num_object_clones":0,"num_object_copies":280258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30215,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":140129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127654,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7583,"num_bytes_recovered":29921299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156241,"ondisk_log_size":156241,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982240,"kb_used_data":1170860,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281133280,"statfs":{"total":289910292480,"available":287880478720,"internally_reserved":0,"allocated":1198960640,"data_stored":1198061004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":830786570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,9,8,5,2,0,2,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24805376,"num_objects":-6056,"num_object_clones":0,"num_object_copies":-12112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":-503,"num_objects_unfound":0,"num_objects_dirty":-6056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6056,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":707,"num_bytes_recovered":2056192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002710"},"pg_stats":[{"pgid":"2.f","version":"48'9734","reported_seq":9883,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:12.937683+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:12.937683+0000","last_peered":"2024-09-15T21:22:12.937683+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:12.937683+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35954688,"num_objects":8778,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9734,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5445},{"shards":"0,2","objects":2889}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9857","reported_seq":9958,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.254136+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:14.254136+0000","last_peered":"2024-09-15T21:22:14.254136+0000","last_clean":"2024-09-15T21:22:14.254136+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:14.254136+0000","last_undegraded":"2024-09-15T21:22:14.254136+0000","last_fullsized":"2024-09-15T21:22:14.254136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35975168,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9923","reported_seq":16843,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.006484+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:13.006484+0000","last_peered":"2024-09-15T21:22:13.006484+0000","last_clean":"2024-09-15T21:22:13.006484+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:13.006484+0000","last_undegraded":"2024-09-15T21:22:13.006484+0000","last_fullsized":"2024-09-15T21:22:13.006484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9923,"log_dups_size":0,"ondisk_log_size":9923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642840,"num_objects":8947,"num_object_clones":0,"num_object_copies":17894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8646,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9822","reported_seq":9896,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.144393+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:14.144393+0000","last_peered":"2024-09-15T21:22:14.144393+0000","last_clean":"2024-09-15T21:22:14.144393+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:14.144393+0000","last_undegraded":"2024-09-15T21:22:14.144393+0000","last_fullsized":"2024-09-15T21:22:14.144393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9822,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9834","reported_seq":9957,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.005401+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:13.005401+0000","last_peered":"2024-09-15T21:22:13.005401+0000","last_clean":"2024-09-15T21:22:13.005401+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:13.005401+0000","last_undegraded":"2024-09-15T21:22:13.005401+0000","last_fullsized":"2024-09-15T21:22:13.005401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9834,"log_dups_size":0,"ondisk_log_size":9834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7692,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9692","reported_seq":9890,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.029311+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:13.029311+0000","last_peered":"2024-09-15T21:22:13.029311+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:13.029311+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9692,"log_dups_size":0,"ondisk_log_size":9692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35590144,"num_objects":8689,"num_object_clones":0,"num_object_copies":17378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5350},{"shards":"0,2","objects":2872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9953","reported_seq":10132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.002410+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:13.002410+0000","last_peered":"2024-09-15T21:22:13.002410+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:13.002410+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9953,"log_dups_size":0,"ondisk_log_size":9953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5506},{"shards":"0,2","objects":2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9690","reported_seq":9814,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.020254+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:13.020254+0000","last_peered":"2024-09-15T21:22:13.020254+0000","last_clean":"2024-09-15T21:22:13.020254+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:13.020254+0000","last_undegraded":"2024-09-15T21:22:13.020254+0000","last_fullsized":"2024-09-15T21:22:13.020254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9690,"log_dups_size":0,"ondisk_log_size":9690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7475,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9679","reported_seq":10707,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.330670+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:14.330670+0000","last_peered":"2024-09-15T21:22:14.330670+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:14.330670+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8265,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":1814528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2766},{"shards":"2","objects":5807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9532","reported_seq":9721,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.025237+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:13.025237+0000","last_peered":"2024-09-15T21:22:13.025237+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:13.025237+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34824192,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5192},{"shards":"0,2","objects":2830}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9658","reported_seq":9832,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.247640+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:14.247640+0000","last_peered":"2024-09-15T21:22:14.247640+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:14.247640+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8209,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2753},{"shards":"2","objects":5391}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9799","reported_seq":9954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.288081+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:14.288081+0000","last_peered":"2024-09-15T21:22:14.288081+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:14.288081+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35835904,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8198,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2828},{"shards":"2","objects":5417}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9850","reported_seq":10060,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.011344+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:13.011344+0000","last_peered":"2024-09-15T21:22:13.011344+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:13.011344+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36290560,"num_objects":8860,"num_object_clones":0,"num_object_copies":17720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7157,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9713","reported_seq":9887,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:12.856275+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:12.856275+0000","last_peered":"2024-09-15T21:22:12.856275+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:12.856275+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35631104,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5340},{"shards":"0,2","objects":2876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9669","reported_seq":9871,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.008573+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:13.008573+0000","last_peered":"2024-09-15T21:22:13.008573+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:13.008573+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9669,"log_dups_size":0,"ondisk_log_size":9669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5381},{"shards":"0,2","objects":2887}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9833","reported_seq":16922,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:12.939606+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:12.939606+0000","last_peered":"2024-09-15T21:22:12.939606+0000","last_clean":"2024-09-15T21:22:12.939606+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:12.939606+0000","last_undegraded":"2024-09-15T21:22:12.939606+0000","last_fullsized":"2024-09-15T21:22:12.939606+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983977,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":573952641,"num_objects":140127,"num_object_clones":0,"num_object_copies":280254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30215,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":140127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127652,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7582,"num_bytes_recovered":29921280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1196658688,"data_stored":1196577485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156238,"ondisk_log_size":156238,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642516,"kb_used_data":390084,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93729324,"statfs":{"total":96636764160,"available":95978827776,"internally_reserved":0,"allocated":399446016,"data_stored":399165654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,4,5,2,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789580,"kb_used_data":474744,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93582260,"statfs":{"total":96636764160,"available":95828234240,"internally_reserved":0,"allocated":486137856,"data_stored":485861566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":322367153},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,9,4,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550144,"kb_used_data":306032,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93821696,"statfs":{"total":96636764160,"available":96073416704,"internally_reserved":0,"allocated":313376768,"data_stored":313033784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":249949868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":485412864,"data_stored":485409385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312934400,"data_stored":312864227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398311424,"data_stored":398303873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:14.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:14.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'9734", 'reported_seq': 9883, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:12.937683+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:12.937683+0000', 'last_peered': '2024-09-15T21:22:12.937683+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:12.937683+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9734, 'log_dups_size': 0, 'ondisk_log_size': 9734, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35954688, 'num_objects': 8778, 'num_object_clones': 0, 'num_object_copies': 17556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3333, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9734, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 34, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5445}, {'shards': '0,2', 'objects': 2889}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'9692", 'reported_seq': 9890, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.029311+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:13.029311+0000', 'last_peered': '2024-09-15T21:22:13.029311+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:13.029311+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9692, 'log_dups_size': 0, 'ondisk_log_size': 9692, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35590144, 'num_objects': 8689, 'num_object_clones': 0, 'num_object_copies': 17378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7117, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5350}, {'shards': '0,2', 'objects': 2872}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'9953", 'reported_seq': 10132, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.002410+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:13.002410+0000', 'last_peered': '2024-09-15T21:22:13.002410+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:13.002410+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 9953, 'log_dups_size': 0, 'ondisk_log_size': 9953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36663296, 'num_objects': 8951, 'num_object_clones': 0, 'num_object_copies': 17902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7589, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5506}, {'shards': '0,2', 'objects': 2971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'9679", 'reported_seq': 10707, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:14.330670+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:14.330670+0000', 'last_peered': '2024-09-15T21:22:14.330670+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:14.330670+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9679, 'log_dups_size': 0, 'ondisk_log_size': 9679, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35618816, 'num_objects': 8696, 'num_object_clones': 0, 'num_object_copies': 17392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 2766, 'num_objects_unfound': 0, 'num_objects_dirty': 8696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8265, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 449, 'num_bytes_recovered': 1814528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2766}, {'shards': '2', 'objects': 5807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'9532", 'reported_seq': 9721, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.025237+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:13.025237+0000', 'last_peered': '2024-09-15T21:22:13.025237+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:13.025237+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9532, 'log_dups_size': 0, 'ondisk_log_size': 9532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34824192, 'num_objects': 8502, 'num_object_clones': 0, 'num_object_copies': 17004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3312, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7058, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5192}, {'shards': '0,2', 'objects': 2830}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'9658", 'reported_seq': 9832, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:14.247640+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:14.247640+0000', 'last_peered': '2024-09-15T21:22:14.247640+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:14.247640+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9658, 'log_dups_size': 0, 'ondisk_log_size': 9658, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35336192, 'num_objects': 8627, 'num_object_clones': 0, 'num_object_copies': 17254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3238, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8209, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2753}, {'shards': '2', 'objects': 5391}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:14.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'9799", 'reported_seq': 9954, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:14.288081+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:14.288081+0000', 'last_peered': '2024-09-15T21:22:14.288081+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:14.288081+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9799, 'log_dups_size': 0, 'ondisk_log_size': 9799, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35835904, 'num_objects': 8749, 'num_object_clones': 0, 'num_object_copies': 17498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3334, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8198, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2828}, {'shards': '2', 'objects': 5417}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'9850", 'reported_seq': 10060, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.011344+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:13.011344+0000', 'last_peered': '2024-09-15T21:22:13.011344+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:13.011344+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9850, 'log_dups_size': 0, 'ondisk_log_size': 9850, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36290560, 'num_objects': 8860, 'num_object_clones': 0, 'num_object_copies': 17720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3395, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8860, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7157, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5466}, {'shards': '0,2', 'objects': 2939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'9713", 'reported_seq': 9887, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:12.856275+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:12.856275+0000', 'last_peered': '2024-09-15T21:22:12.856275+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:12.856275+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9713, 'log_dups_size': 0, 'ondisk_log_size': 9713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35631104, 'num_objects': 8699, 'num_object_clones': 0, 'num_object_copies': 17398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3362, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7298, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5340}, {'shards': '0,2', 'objects': 2876}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:14.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'9669", 'reported_seq': 9871, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.008573+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:13.008573+0000', 'last_peered': '2024-09-15T21:22:13.008573+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:13.008573+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9669, 'log_dups_size': 0, 'ondisk_log_size': 9669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35672064, 'num_objects': 8709, 'num_object_clones': 0, 'num_object_copies': 17418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3330, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7316, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5381}, {'shards': '0,2', 'objects': 2887}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:14.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:15.675 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:15.675 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30215/280258 objects degraded (10.781%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":348,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140129,"data_bytes":573952660,"bytes_used":2029813760,"bytes_avail":287880478720,"bytes_total":289910292480,"degraded_objects":30215,"degraded_total":280258,"degraded_ratio":0.10781137380556487,"misplaced_objects":2766,"misplaced_total":280258,"misplaced_ratio":0.0098694774101006928,"recovering_objects_per_sec":58,"recovering_bytes_per_sec":171310,"recovering_keys_per_sec":0,"num_objects_recovered":707,"num_bytes_recovered":2056192,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":504},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:15.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:15.761 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:16.537 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:16.538 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:16.630 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-15T21:22:16.402019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":573952660,"num_objects":140129,"num_object_clones":0,"num_object_copies":280258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30215,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":140129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127654,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7583,"num_bytes_recovered":29921299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156241,"ondisk_log_size":156241,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982240,"kb_used_data":1170860,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281133280,"statfs":{"total":289910292480,"available":287880478720,"internally_reserved":0,"allocated":1198960640,"data_stored":1198061004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":830786570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,9,8,5,2,0,2,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16998400,"num_objects":-4150,"num_object_clones":0,"num_object_copies":-8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4150,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002735"},"pg_stats":[{"pgid":"2.f","version":"48'9734","reported_seq":9883,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:12.937683+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:12.937683+0000","last_peered":"2024-09-15T21:22:12.937683+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:12.937683+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35954688,"num_objects":8778,"num_object_clones":0,"num_object_copies":17556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9734,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5445},{"shards":"0,2","objects":2889}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9857","reported_seq":9958,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.254136+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:14.254136+0000","last_peered":"2024-09-15T21:22:14.254136+0000","last_clean":"2024-09-15T21:22:14.254136+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:14.254136+0000","last_undegraded":"2024-09-15T21:22:14.254136+0000","last_fullsized":"2024-09-15T21:22:14.254136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35975168,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'9923","reported_seq":16843,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.006484+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:13.006484+0000","last_peered":"2024-09-15T21:22:13.006484+0000","last_clean":"2024-09-15T21:22:13.006484+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:13.006484+0000","last_undegraded":"2024-09-15T21:22:13.006484+0000","last_fullsized":"2024-09-15T21:22:13.006484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":9923,"log_dups_size":0,"ondisk_log_size":9923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36642840,"num_objects":8947,"num_object_clones":0,"num_object_copies":17894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8646,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9822","reported_seq":9896,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.144393+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:14.144393+0000","last_peered":"2024-09-15T21:22:14.144393+0000","last_clean":"2024-09-15T21:22:14.144393+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:14.144393+0000","last_undegraded":"2024-09-15T21:22:14.144393+0000","last_fullsized":"2024-09-15T21:22:14.144393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9822,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9834","reported_seq":9957,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.005401+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:13.005401+0000","last_peered":"2024-09-15T21:22:13.005401+0000","last_clean":"2024-09-15T21:22:13.005401+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:13.005401+0000","last_undegraded":"2024-09-15T21:22:13.005401+0000","last_fullsized":"2024-09-15T21:22:13.005401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9834,"log_dups_size":0,"ondisk_log_size":9834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7692,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9692","reported_seq":9890,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.029311+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:13.029311+0000","last_peered":"2024-09-15T21:22:13.029311+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:13.029311+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9692,"log_dups_size":0,"ondisk_log_size":9692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35590144,"num_objects":8689,"num_object_clones":0,"num_object_copies":17378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7117,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5350},{"shards":"0,2","objects":2872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'9953","reported_seq":10132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.002410+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:13.002410+0000","last_peered":"2024-09-15T21:22:13.002410+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:13.002410+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":9953,"log_dups_size":0,"ondisk_log_size":9953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7589,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5506},{"shards":"0,2","objects":2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9690","reported_seq":9814,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.020254+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:13.020254+0000","last_peered":"2024-09-15T21:22:13.020254+0000","last_clean":"2024-09-15T21:22:13.020254+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:13.020254+0000","last_undegraded":"2024-09-15T21:22:13.020254+0000","last_fullsized":"2024-09-15T21:22:13.020254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9690,"log_dups_size":0,"ondisk_log_size":9690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7475,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9679","reported_seq":10707,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.330670+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:14.330670+0000","last_peered":"2024-09-15T21:22:14.330670+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:14.330670+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8265,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":1814528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2766},{"shards":"2","objects":5807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9532","reported_seq":9721,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.025237+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:13.025237+0000","last_peered":"2024-09-15T21:22:13.025237+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:13.025237+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34824192,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5192},{"shards":"0,2","objects":2830}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9658","reported_seq":9832,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.247640+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:14.247640+0000","last_peered":"2024-09-15T21:22:14.247640+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:14.247640+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8209,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2753},{"shards":"2","objects":5391}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9799","reported_seq":9954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.288081+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:14.288081+0000","last_peered":"2024-09-15T21:22:14.288081+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:14.288081+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35835904,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8198,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2828},{"shards":"2","objects":5417}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9850","reported_seq":10060,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.011344+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:13.011344+0000","last_peered":"2024-09-15T21:22:13.011344+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:13.011344+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36290560,"num_objects":8860,"num_object_clones":0,"num_object_copies":17720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7157,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":2939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9713","reported_seq":9887,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:12.856275+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:12.856275+0000","last_peered":"2024-09-15T21:22:12.856275+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:12.856275+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35631104,"num_objects":8699,"num_object_clones":0,"num_object_copies":17398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5340},{"shards":"0,2","objects":2876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9669","reported_seq":9871,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.008573+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:13.008573+0000","last_peered":"2024-09-15T21:22:13.008573+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:13.008573+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9669,"log_dups_size":0,"ondisk_log_size":9669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35672064,"num_objects":8709,"num_object_clones":0,"num_object_copies":17418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7316,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5381},{"shards":"0,2","objects":2887}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9833","reported_seq":16922,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:12.939606+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:12.939606+0000","last_peered":"2024-09-15T21:22:12.939606+0000","last_clean":"2024-09-15T21:22:12.939606+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:12.939606+0000","last_undegraded":"2024-09-15T21:22:12.939606+0000","last_fullsized":"2024-09-15T21:22:12.939606+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983977,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":573952641,"num_objects":140127,"num_object_clones":0,"num_object_copies":280254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30215,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":140127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127652,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7582,"num_bytes_recovered":29921280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1196658688,"data_stored":1196577485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156238,"ondisk_log_size":156238,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642516,"kb_used_data":390084,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93729324,"statfs":{"total":96636764160,"available":95978827776,"internally_reserved":0,"allocated":399446016,"data_stored":399165654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,4,5,2,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789580,"kb_used_data":474744,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93582260,"statfs":{"total":96636764160,"available":95828234240,"internally_reserved":0,"allocated":486137856,"data_stored":485861566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":322367153},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,9,4,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550144,"kb_used_data":306032,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93821696,"statfs":{"total":96636764160,"available":96073416704,"internally_reserved":0,"allocated":313376768,"data_stored":313033784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":249949868},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":485412864,"data_stored":485409385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312934400,"data_stored":312864227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398311424,"data_stored":398303873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:19.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:20.361 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:20.362 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:20.420 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-15T21:22:18.402471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567947924,"num_objects":138663,"num_object_clones":0,"num_object_copies":277326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30114,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":138663,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129120,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7684,"num_bytes_recovered":29921299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157707,"ondisk_log_size":157707,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966516,"kb_used_data":1163448,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281149004,"statfs":{"total":289910292480,"available":287896580096,"internally_reserved":0,"allocated":1191370752,"data_stored":1190471116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":822266890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,2,8,2,25,11,2,1,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23003136,"num_objects":-5616,"num_object_clones":0,"num_object_copies":-11232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-273,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-5616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5616,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":548,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"2.f","version":"48'9854","reported_seq":10037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.051837+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:18.051837+0000","last_peered":"2024-09-15T21:22:18.051837+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:18.051837+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9854,"log_dups_size":0,"ondisk_log_size":9854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9857","reported_seq":9958,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.254136+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:14.254136+0000","last_peered":"2024-09-15T21:22:14.254136+0000","last_clean":"2024-09-15T21:22:14.254136+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:14.254136+0000","last_undegraded":"2024-09-15T21:22:14.254136+0000","last_fullsized":"2024-09-15T21:22:14.254136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35975168,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8583,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10055","reported_seq":16975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.029481+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:18.029481+0000","last_peered":"2024-09-15T21:22:18.029481+0000","last_clean":"2024-09-15T21:22:18.029481+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:18.029481+0000","last_undegraded":"2024-09-15T21:22:18.029481+0000","last_fullsized":"2024-09-15T21:22:18.029481+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36102168,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8778,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9822","reported_seq":9896,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:14.144393+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:14.144393+0000","last_peered":"2024-09-15T21:22:14.144393+0000","last_clean":"2024-09-15T21:22:14.144393+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:14.144393+0000","last_undegraded":"2024-09-15T21:22:14.144393+0000","last_fullsized":"2024-09-15T21:22:14.144393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9822,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9965","reported_seq":10088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.057470+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:18.057470+0000","last_peered":"2024-09-15T21:22:18.057470+0000","last_clean":"2024-09-15T21:22:18.057470+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:18.057470+0000","last_undegraded":"2024-09-15T21:22:18.057470+0000","last_fullsized":"2024-09-15T21:22:18.057470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9965,"log_dups_size":0,"ondisk_log_size":9965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35872768,"num_objects":8758,"num_object_clones":0,"num_object_copies":17516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9833","reported_seq":10071,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.015353+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:18.015353+0000","last_peered":"2024-09-15T21:22:18.015353+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:18.015353+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35012608,"num_objects":8548,"num_object_clones":0,"num_object_copies":17096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5229},{"shards":"0,2","objects":2731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10071","reported_seq":10268,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.056526+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:18.056526+0000","last_peered":"2024-09-15T21:22:18.056526+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:18.056526+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9839","reported_seq":9962,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.034929+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:18.034929+0000","last_peered":"2024-09-15T21:22:18.034929+0000","last_clean":"2024-09-15T21:22:18.034929+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:18.034929+0000","last_undegraded":"2024-09-15T21:22:18.034929+0000","last_fullsized":"2024-09-15T21:22:18.034929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7624,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9679","reported_seq":10707,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.330670+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:14.330670+0000","last_peered":"2024-09-15T21:22:14.330670+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:14.330670+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8265,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":1814528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2766},{"shards":"2","objects":5807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9661","reported_seq":9876,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.027102+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:18.027102+0000","last_peered":"2024-09-15T21:22:18.027102+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:18.027102+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7187,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5076},{"shards":"0,2","objects":2701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9658","reported_seq":9832,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.247640+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:14.247640+0000","last_peered":"2024-09-15T21:22:14.247640+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:14.247640+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9658,"log_dups_size":0,"ondisk_log_size":9658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8209,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2753},{"shards":"2","objects":5391}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9799","reported_seq":9954,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:14.288081+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:14.288081+0000","last_peered":"2024-09-15T21:22:14.288081+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:14.288081+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35835904,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8198,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2828},{"shards":"2","objects":5417}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9988","reported_seq":10222,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.089472+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:18.089472+0000","last_peered":"2024-09-15T21:22:18.089472+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:18.089472+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9988,"log_dups_size":0,"ondisk_log_size":9988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7295,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5340},{"shards":"0,2","objects":2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9857","reported_seq":10061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:17.992689+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:17.992689+0000","last_peered":"2024-09-15T21:22:17.992689+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:17.992689+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":2732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9809","reported_seq":10041,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.092391+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:18.092391+0000","last_peered":"2024-09-15T21:22:18.092391+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:18.092391+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9809,"log_dups_size":0,"ondisk_log_size":9809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5256},{"shards":"0,2","objects":2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9957","reported_seq":17046,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:17.875349+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:17.875349+0000","last_peered":"2024-09-15T21:22:17.875349+0000","last_clean":"2024-09-15T21:22:17.875349+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:17.875349+0000","last_undegraded":"2024-09-15T21:22:17.875349+0000","last_fullsized":"2024-09-15T21:22:17.875349+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476073,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567947905,"num_objects":138661,"num_object_clones":0,"num_object_copies":277322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30114,"num_objects_misplaced":2766,"num_objects_unfound":0,"num_objects_dirty":138661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129118,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7683,"num_bytes_recovered":29921280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188966400,"data_stored":1188885197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157704,"ondisk_log_size":157704,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953532,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642516,"kb_used_data":390084,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93729324,"statfs":{"total":96636764160,"available":95978827776,"internally_reserved":0,"allocated":399446016,"data_stored":399165654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,4,5,2,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770736,"kb_used_data":468372,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93601104,"statfs":{"total":96636764160,"available":95847530496,"internally_reserved":0,"allocated":479612928,"data_stored":479336638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,0,18,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553264,"kb_used_data":304992,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93818576,"statfs":{"total":96636764160,"available":96070221824,"internally_reserved":0,"allocated":312311808,"data_stored":311968824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478797824,"data_stored":478794345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311857152,"data_stored":311786979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":398311424,"data_stored":398303873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:20.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:21.283 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:21.358 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:22:20.408255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565224084,"num_objects":137998,"num_object_clones":0,"num_object_copies":275996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30093,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":137998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129785,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":30441491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158372,"ondisk_log_size":158372,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966460,"kb_used_data":1159228,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281149060,"statfs":{"total":289910292480,"available":287896637440,"internally_reserved":0,"allocated":1187049472,"data_stored":1186149836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":826526730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,3,7,5,22,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19996672,"num_objects":-4882,"num_object_clones":0,"num_object_copies":-9764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":-402,"num_objects_unfound":0,"num_objects_dirty":-4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":608,"num_bytes_recovered":1646592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008101"},"pg_stats":[{"pgid":"2.f","version":"48'9854","reported_seq":10037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.051837+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:18.051837+0000","last_peered":"2024-09-15T21:22:18.051837+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:18.051837+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9854,"log_dups_size":0,"ondisk_log_size":9854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9994","reported_seq":10095,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:19.283285+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:19.283285+0000","last_peered":"2024-09-15T21:22:19.283285+0000","last_clean":"2024-09-15T21:22:19.283285+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:19.283285+0000","last_undegraded":"2024-09-15T21:22:19.283285+0000","last_fullsized":"2024-09-15T21:22:19.283285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9994,"log_dups_size":0,"ondisk_log_size":9994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8720,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10055","reported_seq":16975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.029481+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:18.029481+0000","last_peered":"2024-09-15T21:22:18.029481+0000","last_clean":"2024-09-15T21:22:18.029481+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:18.029481+0000","last_undegraded":"2024-09-15T21:22:18.029481+0000","last_fullsized":"2024-09-15T21:22:18.029481+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36102168,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8778,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9953","reported_seq":10027,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:19.310593+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:19.310593+0000","last_peered":"2024-09-15T21:22:19.310593+0000","last_clean":"2024-09-15T21:22:19.310593+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:19.310593+0000","last_undegraded":"2024-09-15T21:22:19.310593+0000","last_fullsized":"2024-09-15T21:22:19.310593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9953,"log_dups_size":0,"ondisk_log_size":9953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9953,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9965","reported_seq":10088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.057470+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:18.057470+0000","last_peered":"2024-09-15T21:22:18.057470+0000","last_clean":"2024-09-15T21:22:18.057470+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:18.057470+0000","last_undegraded":"2024-09-15T21:22:18.057470+0000","last_fullsized":"2024-09-15T21:22:18.057470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9965,"log_dups_size":0,"ondisk_log_size":9965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35872768,"num_objects":8758,"num_object_clones":0,"num_object_copies":17516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9833","reported_seq":10071,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.015353+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:18.015353+0000","last_peered":"2024-09-15T21:22:18.015353+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:18.015353+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35012608,"num_objects":8548,"num_object_clones":0,"num_object_copies":17096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5229},{"shards":"0,2","objects":2731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10071","reported_seq":10268,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.056526+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:18.056526+0000","last_peered":"2024-09-15T21:22:18.056526+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:18.056526+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9839","reported_seq":9962,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.034929+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:18.034929+0000","last_peered":"2024-09-15T21:22:18.034929+0000","last_clean":"2024-09-15T21:22:18.034929+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:18.034929+0000","last_undegraded":"2024-09-15T21:22:18.034929+0000","last_fullsized":"2024-09-15T21:22:18.034929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7624,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9823","reported_seq":11105,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.344044+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:19.344044+0000","last_peered":"2024-09-15T21:22:19.344044+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:19.344044+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9823,"log_dups_size":0,"ondisk_log_size":9823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35028992,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":2334720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2639},{"shards":"2","objects":5790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9661","reported_seq":9876,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.027102+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:18.027102+0000","last_peered":"2024-09-15T21:22:18.027102+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:18.027102+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7187,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5076},{"shards":"0,2","objects":2701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9781","reported_seq":9975,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.281094+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:19.281094+0000","last_peered":"2024-09-15T21:22:19.281094+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:19.281094+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9781,"log_dups_size":0,"ondisk_log_size":9781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8332,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2630},{"shards":"2","objects":5278}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9929","reported_seq":10106,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.270754+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:19.270754+0000","last_peered":"2024-09-15T21:22:19.270754+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:19.270754+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9929,"log_dups_size":0,"ondisk_log_size":9929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35303424,"num_objects":8619,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8328,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2698},{"shards":"2","objects":5298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9988","reported_seq":10222,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.089472+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:18.089472+0000","last_peered":"2024-09-15T21:22:18.089472+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:18.089472+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9988,"log_dups_size":0,"ondisk_log_size":9988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7295,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5340},{"shards":"0,2","objects":2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9857","reported_seq":10061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:17.992689+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:17.992689+0000","last_peered":"2024-09-15T21:22:17.992689+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:17.992689+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":2732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9809","reported_seq":10041,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.092391+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:18.092391+0000","last_peered":"2024-09-15T21:22:18.092391+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:18.092391+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9809,"log_dups_size":0,"ondisk_log_size":9809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5256},{"shards":"0,2","objects":2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9957","reported_seq":17046,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:17.875349+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:17.875349+0000","last_peered":"2024-09-15T21:22:17.875349+0000","last_clean":"2024-09-15T21:22:17.875349+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:17.875349+0000","last_undegraded":"2024-09-15T21:22:17.875349+0000","last_fullsized":"2024-09-15T21:22:17.875349+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476073,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565224065,"num_objects":137996,"num_object_clones":0,"num_object_copies":275992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30093,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":137996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129783,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7831,"num_bytes_recovered":30441472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1184530432,"data_stored":1184449229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158369,"ondisk_log_size":158369,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953533,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642460,"kb_used_data":385864,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93729380,"statfs":{"total":96636764160,"available":95978885120,"internally_reserved":0,"allocated":395124736,"data_stored":394844374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770736,"kb_used_data":468372,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93601104,"statfs":{"total":96636764160,"available":95847530496,"internally_reserved":0,"allocated":479612928,"data_stored":479336638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,0,18,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553264,"kb_used_data":304992,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93818576,"statfs":{"total":96636764160,"available":96070221824,"internally_reserved":0,"allocated":312311808,"data_stored":311968824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478797824,"data_stored":478794345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311857152,"data_stored":311786979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393875456,"data_stored":393867905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'9854", 'reported_seq': 10037, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.051837+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:18.051837+0000', 'last_peered': '2024-09-15T21:22:18.051837+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:18.051837+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 9854, 'log_dups_size': 0, 'ondisk_log_size': 9854, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35463168, 'num_objects': 8658, 'num_object_clones': 0, 'num_object_copies': 17316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3316, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9854, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5342}, {'shards': '0,2', 'objects': 2769}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'9833", 'reported_seq': 10071, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.015353+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:18.015353+0000', 'last_peered': '2024-09-15T21:22:18.015353+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:18.015353+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9833, 'log_dups_size': 0, 'ondisk_log_size': 9833, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35012608, 'num_objects': 8548, 'num_object_clones': 0, 'num_object_copies': 17096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3321, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7258, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5229}, {'shards': '0,2', 'objects': 2731}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10071", 'reported_seq': 10268, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.056526+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:18.056526+0000', 'last_peered': '2024-09-15T21:22:18.056526+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:18.056526+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36179968, 'num_objects': 8833, 'num_object_clones': 0, 'num_object_copies': 17666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7707, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5397}, {'shards': '0,2', 'objects': 2853}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'9823", 'reported_seq': 11105, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:19.344044+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:19.344044+0000', 'last_peered': '2024-09-15T21:22:19.344044+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:19.344044+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9823, 'log_dups_size': 0, 'ondisk_log_size': 9823, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35028992, 'num_objects': 8552, 'num_object_clones': 0, 'num_object_copies': 17104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 2639, 'num_objects_unfound': 0, 'num_objects_dirty': 8552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8409, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 576, 'num_bytes_recovered': 2334720, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2639}, {'shards': '2', 'objects': 5790}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:21.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'9661", 'reported_seq': 9876, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.027102+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:18.027102+0000', 'last_peered': '2024-09-15T21:22:18.027102+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:18.027102+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9661, 'log_dups_size': 0, 'ondisk_log_size': 9661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34295808, 'num_objects': 8373, 'num_object_clones': 0, 'num_object_copies': 16746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3299, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7187, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5076}, {'shards': '0,2', 'objects': 2701}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'9781", 'reported_seq': 9975, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:19.281094+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:19.281094+0000', 'last_peered': '2024-09-15T21:22:19.281094+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:19.281094+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9781, 'log_dups_size': 0, 'ondisk_log_size': 9781, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34832384, 'num_objects': 8504, 'num_object_clones': 0, 'num_object_copies': 17008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3228, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8332, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2630}, {'shards': '2', 'objects': 5278}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'9929", 'reported_seq': 10106, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:19.270754+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:19.270754+0000', 'last_peered': '2024-09-15T21:22:19.270754+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:19.270754+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 9929, 'log_dups_size': 0, 'ondisk_log_size': 9929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35303424, 'num_objects': 8619, 'num_object_clones': 0, 'num_object_copies': 17238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8328, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2698}, {'shards': '2', 'objects': 5298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'9988", 'reported_seq': 10222, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.089472+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:18.089472+0000', 'last_peered': '2024-09-15T21:22:18.089472+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:18.089472+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 9988, 'log_dups_size': 0, 'ondisk_log_size': 9988, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35725312, 'num_objects': 8722, 'num_object_clones': 0, 'num_object_copies': 17444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3383, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7295, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5340}, {'shards': '0,2', 'objects': 2801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'9857", 'reported_seq': 10061, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:17.992689+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:17.992689+0000', 'last_peered': '2024-09-15T21:22:17.992689+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:17.992689+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9857, 'log_dups_size': 0, 'ondisk_log_size': 9857, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35041280, 'num_objects': 8555, 'num_object_clones': 0, 'num_object_copies': 17110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3347, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7442, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5211}, {'shards': '0,2', 'objects': 2732}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:21.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'9809", 'reported_seq': 10041, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.092391+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:18.092391+0000', 'last_peered': '2024-09-15T21:22:18.092391+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:18.092391+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9809, 'log_dups_size': 0, 'ondisk_log_size': 9809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35098624, 'num_objects': 8569, 'num_object_clones': 0, 'num_object_copies': 17138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3315, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7456, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5256}, {'shards': '0,2', 'objects': 2747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:21.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:22.274 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30093/275996 objects degraded (10.903%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":355,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137998,"data_bytes":565224084,"bytes_used":2013655040,"bytes_avail":287896637440,"bytes_total":289910292480,"degraded_objects":30093,"degraded_total":275996,"degraded_ratio":0.10903418890128842,"misplaced_objects":2639,"misplaced_total":275996,"misplaced_ratio":0.0095617327787359246,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":137123,"recovering_keys_per_sec":0,"num_objects_recovered":608,"num_bytes_recovered":1646592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:22.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:22.315 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:22.955 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:22.956 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:23.009 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-15T21:22:22.412409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565224084,"num_objects":137998,"num_object_clones":0,"num_object_copies":275996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30093,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":137998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129785,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":30441491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158372,"ondisk_log_size":158372,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1966460,"kb_used_data":1159228,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281149060,"statfs":{"total":289910292480,"available":287896637440,"internally_reserved":0,"allocated":1187049472,"data_stored":1186149836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":826526730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,3,7,5,22,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17424384,"num_objects":-4254,"num_object_clones":0,"num_object_copies":-8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011728"},"pg_stats":[{"pgid":"2.f","version":"48'9854","reported_seq":10037,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.051837+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:18.051837+0000","last_peered":"2024-09-15T21:22:18.051837+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:18.051837+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9854,"log_dups_size":0,"ondisk_log_size":9854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35463168,"num_objects":8658,"num_object_clones":0,"num_object_copies":17316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9854,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'9994","reported_seq":10095,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:19.283285+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:19.283285+0000","last_peered":"2024-09-15T21:22:19.283285+0000","last_clean":"2024-09-15T21:22:19.283285+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:19.283285+0000","last_undegraded":"2024-09-15T21:22:19.283285+0000","last_fullsized":"2024-09-15T21:22:19.283285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":9994,"log_dups_size":0,"ondisk_log_size":9994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8720,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10055","reported_seq":16975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.029481+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:18.029481+0000","last_peered":"2024-09-15T21:22:18.029481+0000","last_clean":"2024-09-15T21:22:18.029481+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:18.029481+0000","last_undegraded":"2024-09-15T21:22:18.029481+0000","last_fullsized":"2024-09-15T21:22:18.029481+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36102168,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8778,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'9953","reported_seq":10027,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:19.310593+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:19.310593+0000","last_peered":"2024-09-15T21:22:19.310593+0000","last_clean":"2024-09-15T21:22:19.310593+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:19.310593+0000","last_undegraded":"2024-09-15T21:22:19.310593+0000","last_fullsized":"2024-09-15T21:22:19.310593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":9953,"log_dups_size":0,"ondisk_log_size":9953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9953,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'9965","reported_seq":10088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.057470+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:18.057470+0000","last_peered":"2024-09-15T21:22:18.057470+0000","last_clean":"2024-09-15T21:22:18.057470+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:18.057470+0000","last_undegraded":"2024-09-15T21:22:18.057470+0000","last_fullsized":"2024-09-15T21:22:18.057470+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":9965,"log_dups_size":0,"ondisk_log_size":9965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35872768,"num_objects":8758,"num_object_clones":0,"num_object_copies":17516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9833","reported_seq":10071,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.015353+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:18.015353+0000","last_peered":"2024-09-15T21:22:18.015353+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:18.015353+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35012608,"num_objects":8548,"num_object_clones":0,"num_object_copies":17096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7258,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5229},{"shards":"0,2","objects":2731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10071","reported_seq":10268,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.056526+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:18.056526+0000","last_peered":"2024-09-15T21:22:18.056526+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:18.056526+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7707,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9839","reported_seq":9962,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.034929+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:18.034929+0000","last_peered":"2024-09-15T21:22:18.034929+0000","last_clean":"2024-09-15T21:22:18.034929+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:18.034929+0000","last_undegraded":"2024-09-15T21:22:18.034929+0000","last_fullsized":"2024-09-15T21:22:18.034929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7624,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9823","reported_seq":11105,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.344044+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:19.344044+0000","last_peered":"2024-09-15T21:22:19.344044+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:19.344044+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9823,"log_dups_size":0,"ondisk_log_size":9823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35028992,"num_objects":8552,"num_object_clones":0,"num_object_copies":17104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":8552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":2334720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2639},{"shards":"2","objects":5790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9661","reported_seq":9876,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.027102+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:18.027102+0000","last_peered":"2024-09-15T21:22:18.027102+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:18.027102+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9661,"log_dups_size":0,"ondisk_log_size":9661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7187,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5076},{"shards":"0,2","objects":2701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9781","reported_seq":9975,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.281094+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:19.281094+0000","last_peered":"2024-09-15T21:22:19.281094+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:19.281094+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9781,"log_dups_size":0,"ondisk_log_size":9781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8332,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2630},{"shards":"2","objects":5278}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'9929","reported_seq":10106,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:19.270754+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:19.270754+0000","last_peered":"2024-09-15T21:22:19.270754+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:19.270754+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":9929,"log_dups_size":0,"ondisk_log_size":9929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35303424,"num_objects":8619,"num_object_clones":0,"num_object_copies":17238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8328,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2698},{"shards":"2","objects":5298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'9988","reported_seq":10222,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.089472+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:18.089472+0000","last_peered":"2024-09-15T21:22:18.089472+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:18.089472+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":9988,"log_dups_size":0,"ondisk_log_size":9988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7295,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5340},{"shards":"0,2","objects":2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9857","reported_seq":10061,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:17.992689+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:17.992689+0000","last_peered":"2024-09-15T21:22:17.992689+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:17.992689+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":2732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9809","reported_seq":10041,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.092391+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:18.092391+0000","last_peered":"2024-09-15T21:22:18.092391+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:18.092391+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9809,"log_dups_size":0,"ondisk_log_size":9809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5256},{"shards":"0,2","objects":2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'9957","reported_seq":17046,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:17.875349+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:17.875349+0000","last_peered":"2024-09-15T21:22:17.875349+0000","last_clean":"2024-09-15T21:22:17.875349+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:17.875349+0000","last_undegraded":"2024-09-15T21:22:17.875349+0000","last_fullsized":"2024-09-15T21:22:17.875349+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":9957,"log_dups_size":0,"ondisk_log_size":9957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476073,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565224065,"num_objects":137996,"num_object_clones":0,"num_object_copies":275992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30093,"num_objects_misplaced":2639,"num_objects_unfound":0,"num_objects_dirty":137996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129783,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7831,"num_bytes_recovered":30441472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1184530432,"data_stored":1184449229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158369,"ondisk_log_size":158369,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953533,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642460,"kb_used_data":385864,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93729380,"statfs":{"total":96636764160,"available":95978885120,"internally_reserved":0,"allocated":395124736,"data_stored":394844374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149760,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770736,"kb_used_data":468372,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93601104,"statfs":{"total":96636764160,"available":95847530496,"internally_reserved":0,"allocated":479612928,"data_stored":479336638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":309587633},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,5,0,18,6,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626478,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553264,"kb_used_data":304992,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93818576,"statfs":{"total":96636764160,"available":96070221824,"internally_reserved":0,"allocated":312311808,"data_stored":311968824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":254209708},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478797824,"data_stored":478794345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311857152,"data_stored":311786979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393875456,"data_stored":393867905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:26.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:26.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:26.414 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:26.449 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-15T21:22:24.412729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556475028,"num_objects":135862,"num_object_clones":0,"num_object_copies":271724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":135862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131921,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8047,"num_bytes_recovered":30998547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160008,"ondisk_log_size":160008,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1954356,"kb_used_data":1147140,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281161164,"statfs":{"total":289910292480,"available":287909031936,"internally_reserved":0,"allocated":1174671360,"data_stored":1173771724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":826526730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,8,6,6,6,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26173440,"num_objects":-6390,"num_object_clones":0,"num_object_copies":-12780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":-390,"num_objects_unfound":0,"num_objects_dirty":-6390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011675"},"pg_stats":[{"pgid":"2.f","version":"48'10013","reported_seq":10226,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.053385+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:23.053385+0000","last_peered":"2024-09-15T21:22:23.053385+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:23.053385+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34811904,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10013,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5198},{"shards":"0,2","objects":2610}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10133","reported_seq":10234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.098823+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:24.098823+0000","last_peered":"2024-09-15T21:22:24.098823+0000","last_clean":"2024-09-15T21:22:24.098823+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:24.098823+0000","last_undegraded":"2024-09-15T21:22:24.098823+0000","last_fullsized":"2024-09-15T21:22:24.098823+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8859,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10185","reported_seq":17105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.110163+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:23.110163+0000","last_peered":"2024-09-15T21:22:23.110163+0000","last_clean":"2024-09-15T21:22:23.110163+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:23.110163+0000","last_undegraded":"2024-09-15T21:22:23.110163+0000","last_fullsized":"2024-09-15T21:22:23.110163+0000","mapping_epoch":45,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35569688,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8908,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10089","reported_seq":10163,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.347707+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:24.347707+0000","last_peered":"2024-09-15T21:22:24.347707+0000","last_clean":"2024-09-15T21:22:24.347707+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:24.347707+0000","last_undegraded":"2024-09-15T21:22:24.347707+0000","last_fullsized":"2024-09-15T21:22:24.347707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34738176,"num_objects":8481,"num_object_clones":0,"num_object_copies":16962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10089,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10102","reported_seq":10225,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.121162+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:23.121162+0000","last_peered":"2024-09-15T21:22:23.121162+0000","last_clean":"2024-09-15T21:22:23.121162+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:23.121162+0000","last_undegraded":"2024-09-15T21:22:23.121162+0000","last_fullsized":"2024-09-15T21:22:23.121162+0000","mapping_epoch":35,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7960,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9951","reported_seq":10207,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.010864+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:23.010864+0000","last_peered":"2024-09-15T21:22:23.010864+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:23.010864+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9951,"log_dups_size":0,"ondisk_log_size":9951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7376,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5120},{"shards":"0,2","objects":2613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10191","reported_seq":10406,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.055877+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:23.055877+0000","last_peered":"2024-09-15T21:22:23.055877+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:23.055877+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35688448,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7827,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5286},{"shards":"0,2","objects":2733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9977","reported_seq":10101,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:22.977310+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:22.977310+0000","last_peered":"2024-09-15T21:22:22.977310+0000","last_clean":"2024-09-15T21:22:22.977310+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:22.977310+0000","last_undegraded":"2024-09-15T21:22:22.977310+0000","last_fullsized":"2024-09-15T21:22:22.977310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7762,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9959","reported_seq":11513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.349218+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:24.349218+0000","last_peered":"2024-09-15T21:22:24.349218+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:24.349218+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9959,"log_dups_size":0,"ondisk_log_size":9959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8545,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":2891776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2503},{"shards":"2","objects":5790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9801","reported_seq":10038,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.097570+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:23.097570+0000","last_peered":"2024-09-15T21:22:23.097570+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:23.097570+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722368,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7327,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4947},{"shards":"0,2","objects":2561}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9910","reported_seq":10114,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.047875+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:24.047875+0000","last_peered":"2024-09-15T21:22:24.047875+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:24.047875+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8461,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10077","reported_seq":10262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.298753+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:24.298753+0000","last_peered":"2024-09-15T21:22:24.298753+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:24.298753+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34697216,"num_objects":8471,"num_object_clones":0,"num_object_copies":16942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2550},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10127","reported_seq":10390,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.054743+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:23.054743+0000","last_peered":"2024-09-15T21:22:23.054743+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:23.054743+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35155968,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7434,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":2662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9981","reported_seq":10189,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.010708+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:23.010708+0000","last_peered":"2024-09-15T21:22:23.010708+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:23.010708+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7566,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5089},{"shards":"0,2","objects":2608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9921","reported_seq":10171,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.020944+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:23.020944+0000","last_peered":"2024-09-15T21:22:23.020944+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:23.020944+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9921,"log_dups_size":0,"ondisk_log_size":9921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34639872,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7568,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5153},{"shards":"0,2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10088","reported_seq":17177,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.016731+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:23.016731+0000","last_peered":"2024-09-15T21:22:23.016731+0000","last_clean":"2024-09-15T21:22:23.016731+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:23.016731+0000","last_undegraded":"2024-09-15T21:22:23.016731+0000","last_fullsized":"2024-09-15T21:22:23.016731+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34939497,"num_objects":8531,"num_object_clones":0,"num_object_copies":17062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556475009,"num_objects":135860,"num_object_clones":0,"num_object_copies":271720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":135860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131919,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8046,"num_bytes_recovered":30998528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172430848,"data_stored":1172349645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160005,"ondisk_log_size":160005,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625480,"kb_used_data":381368,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93746360,"statfs":{"total":96636764160,"available":95996272640,"internally_reserved":0,"allocated":390520832,"data_stored":390240470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149761,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768448,"kb_used_data":461936,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93603392,"statfs":{"total":96636764160,"available":95849873408,"internally_reserved":0,"allocated":473022464,"data_stored":472746174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,2,2,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560428,"kb_used_data":303836,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93811412,"statfs":{"total":96636764160,"available":96062885888,"internally_reserved":0,"allocated":311128064,"data_stored":310785080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":472285184,"data_stored":472281705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310677504,"data_stored":310607331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389468160,"data_stored":389460609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:26.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:27.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:27.183 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:27.257 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-15T21:22:26.413226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556475028,"num_objects":135862,"num_object_clones":0,"num_object_copies":271724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":135862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131921,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8047,"num_bytes_recovered":30998547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160008,"ondisk_log_size":160008,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1954356,"kb_used_data":1147140,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281161164,"statfs":{"total":289910292480,"available":287909031936,"internally_reserved":0,"allocated":1174671360,"data_stored":1173771724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":826526730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,8,6,6,6,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17477632,"num_objects":-4267,"num_object_clones":0,"num_object_copies":-8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4267,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":464,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011711"},"pg_stats":[{"pgid":"2.f","version":"48'10013","reported_seq":10226,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.053385+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:23.053385+0000","last_peered":"2024-09-15T21:22:23.053385+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:23.053385+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34811904,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10013,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5198},{"shards":"0,2","objects":2610}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10133","reported_seq":10234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.098823+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:24.098823+0000","last_peered":"2024-09-15T21:22:24.098823+0000","last_clean":"2024-09-15T21:22:24.098823+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:24.098823+0000","last_undegraded":"2024-09-15T21:22:24.098823+0000","last_fullsized":"2024-09-15T21:22:24.098823+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8859,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10185","reported_seq":17105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.110163+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:23.110163+0000","last_peered":"2024-09-15T21:22:23.110163+0000","last_clean":"2024-09-15T21:22:23.110163+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:23.110163+0000","last_undegraded":"2024-09-15T21:22:23.110163+0000","last_fullsized":"2024-09-15T21:22:23.110163+0000","mapping_epoch":45,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35569688,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8908,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10089","reported_seq":10163,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.347707+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:24.347707+0000","last_peered":"2024-09-15T21:22:24.347707+0000","last_clean":"2024-09-15T21:22:24.347707+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:24.347707+0000","last_undegraded":"2024-09-15T21:22:24.347707+0000","last_fullsized":"2024-09-15T21:22:24.347707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34738176,"num_objects":8481,"num_object_clones":0,"num_object_copies":16962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10089,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10102","reported_seq":10225,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.121162+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:23.121162+0000","last_peered":"2024-09-15T21:22:23.121162+0000","last_clean":"2024-09-15T21:22:23.121162+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:23.121162+0000","last_undegraded":"2024-09-15T21:22:23.121162+0000","last_fullsized":"2024-09-15T21:22:23.121162+0000","mapping_epoch":35,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35311616,"num_objects":8621,"num_object_clones":0,"num_object_copies":17242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7960,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'9951","reported_seq":10207,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.010864+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:23.010864+0000","last_peered":"2024-09-15T21:22:23.010864+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:23.010864+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":9951,"log_dups_size":0,"ondisk_log_size":9951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7376,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5120},{"shards":"0,2","objects":2613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10191","reported_seq":10406,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.055877+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:23.055877+0000","last_peered":"2024-09-15T21:22:23.055877+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:23.055877+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35688448,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7827,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5286},{"shards":"0,2","objects":2733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'9977","reported_seq":10101,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:22.977310+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:22.977310+0000","last_peered":"2024-09-15T21:22:22.977310+0000","last_clean":"2024-09-15T21:22:22.977310+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:22.977310+0000","last_undegraded":"2024-09-15T21:22:22.977310+0000","last_fullsized":"2024-09-15T21:22:22.977310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":9977,"log_dups_size":0,"ondisk_log_size":9977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7762,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9959","reported_seq":11513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.349218+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:24.349218+0000","last_peered":"2024-09-15T21:22:24.349218+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:24.349218+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9959,"log_dups_size":0,"ondisk_log_size":9959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8545,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":2891776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2503},{"shards":"2","objects":5790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9801","reported_seq":10038,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.097570+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:23.097570+0000","last_peered":"2024-09-15T21:22:23.097570+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:23.097570+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722368,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7327,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4947},{"shards":"0,2","objects":2561}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9910","reported_seq":10114,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.047875+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:24.047875+0000","last_peered":"2024-09-15T21:22:24.047875+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:24.047875+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8461,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10077","reported_seq":10262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.298753+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:24.298753+0000","last_peered":"2024-09-15T21:22:24.298753+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:24.298753+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34697216,"num_objects":8471,"num_object_clones":0,"num_object_copies":16942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2550},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10127","reported_seq":10390,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.054743+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:23.054743+0000","last_peered":"2024-09-15T21:22:23.054743+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:23.054743+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35155968,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7434,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":2662}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'9981","reported_seq":10189,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.010708+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:23.010708+0000","last_peered":"2024-09-15T21:22:23.010708+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:23.010708+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":9981,"log_dups_size":0,"ondisk_log_size":9981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7566,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5089},{"shards":"0,2","objects":2608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'9921","reported_seq":10171,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.020944+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:23.020944+0000","last_peered":"2024-09-15T21:22:23.020944+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:23.020944+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":9921,"log_dups_size":0,"ondisk_log_size":9921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34639872,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7568,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5153},{"shards":"0,2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10088","reported_seq":17177,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.016731+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:23.016731+0000","last_peered":"2024-09-15T21:22:23.016731+0000","last_clean":"2024-09-15T21:22:23.016731+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:23.016731+0000","last_undegraded":"2024-09-15T21:22:23.016731+0000","last_fullsized":"2024-09-15T21:22:23.016731+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34939497,"num_objects":8531,"num_object_clones":0,"num_object_copies":17062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7748,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556475009,"num_objects":135860,"num_object_clones":0,"num_object_copies":271720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30014,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":135860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131919,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8046,"num_bytes_recovered":30998528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172430848,"data_stored":1172349645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160005,"ondisk_log_size":160005,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625480,"kb_used_data":381368,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93746360,"statfs":{"total":96636764160,"available":95996272640,"internally_reserved":0,"allocated":390520832,"data_stored":390240470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149761,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768448,"kb_used_data":461936,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93603392,"statfs":{"total":96636764160,"available":95849873408,"internally_reserved":0,"allocated":473022464,"data_stored":472746174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":313847473},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,2,2,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626479,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560428,"kb_used_data":303836,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93811412,"statfs":{"total":96636764160,"available":96062885888,"internally_reserved":0,"allocated":311128064,"data_stored":310785080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":262729388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":472285184,"data_stored":472281705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310677504,"data_stored":310607331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389468160,"data_stored":389460609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10013", 'reported_seq': 10226, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.053385+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:23.053385+0000', 'last_peered': '2024-09-15T21:22:23.053385+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:23.053385+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34811904, 'num_objects': 8499, 'num_object_clones': 0, 'num_object_copies': 16998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3301, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10013, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5198}, {'shards': '0,2', 'objects': 2610}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'9951", 'reported_seq': 10207, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.010864+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:23.010864+0000', 'last_peered': '2024-09-15T21:22:23.010864+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:23.010864+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 9951, 'log_dups_size': 0, 'ondisk_log_size': 9951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34529280, 'num_objects': 8430, 'num_object_clones': 0, 'num_object_copies': 16860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3312, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7376, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5120}, {'shards': '0,2', 'objects': 2613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10191", 'reported_seq': 10406, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.055877+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:23.055877+0000', 'last_peered': '2024-09-15T21:22:23.055877+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:23.055877+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35688448, 'num_objects': 8713, 'num_object_clones': 0, 'num_object_copies': 17426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3429, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7827, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5286}, {'shards': '0,2', 'objects': 2733}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:27.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'9959", 'reported_seq': 11513, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:24.349218+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:24.349218+0000', 'last_peered': '2024-09-15T21:22:24.349218+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:24.349218+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 9959, 'log_dups_size': 0, 'ondisk_log_size': 9959, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34471936, 'num_objects': 8416, 'num_object_clones': 0, 'num_object_copies': 16832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 2503, 'num_objects_unfound': 0, 'num_objects_dirty': 8416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8545, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 712, 'num_bytes_recovered': 2891776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2503}, {'shards': '2', 'objects': 5790}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'9801", 'reported_seq': 10038, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.097570+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:23.097570+0000', 'last_peered': '2024-09-15T21:22:23.097570+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:23.097570+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9801, 'log_dups_size': 0, 'ondisk_log_size': 9801, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33722368, 'num_objects': 8233, 'num_object_clones': 0, 'num_object_copies': 16466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3288, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7327, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4947}, {'shards': '0,2', 'objects': 2561}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'9910", 'reported_seq': 10114, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:24.047875+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:24.047875+0000', 'last_peered': '2024-09-15T21:22:24.047875+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:24.047875+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 9910, 'log_dups_size': 0, 'ondisk_log_size': 9910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34304000, 'num_objects': 8375, 'num_object_clones': 0, 'num_object_copies': 16750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3223, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8461, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2501}, {'shards': '2', 'objects': 5154}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10077", 'reported_seq': 10262, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:24.298753+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:24.298753+0000', 'last_peered': '2024-09-15T21:22:24.298753+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:24.298753+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34697216, 'num_objects': 8471, 'num_object_clones': 0, 'num_object_copies': 16942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3319, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8476, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2550}, {'shards': '2', 'objects': 5154}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10127", 'reported_seq': 10390, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.054743+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:23.054743+0000', 'last_peered': '2024-09-15T21:22:23.054743+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:23.054743+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35155968, 'num_objects': 8583, 'num_object_clones': 0, 'num_object_copies': 17166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3368, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7434, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5216}, {'shards': '0,2', 'objects': 2662}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:27.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'9981", 'reported_seq': 10189, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.010708+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:23.010708+0000', 'last_peered': '2024-09-15T21:22:23.010708+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:23.010708+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 9981, 'log_dups_size': 0, 'ondisk_log_size': 9981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34533376, 'num_objects': 8431, 'num_object_clones': 0, 'num_object_copies': 16862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3345, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7566, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5089}, {'shards': '0,2', 'objects': 2608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:27.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'9921", 'reported_seq': 10171, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:23.020944+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:23.020944+0000', 'last_peered': '2024-09-15T21:22:23.020944+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:23.020944+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 9921, 'log_dups_size': 0, 'ondisk_log_size': 9921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34639872, 'num_objects': 8457, 'num_object_clones': 0, 'num_object_copies': 16914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7568, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5153}, {'shards': '0,2', 'objects': 2635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:27.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:28.256 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30014/271724 objects degraded (11.046%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":361,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135862,"data_bytes":556475028,"bytes_used":2001260544,"bytes_avail":287909031936,"bytes_total":289910292480,"degraded_objects":30014,"degraded_total":271724,"degraded_ratio":0.11045767028308136,"misplaced_objects":2503,"misplaced_total":271724,"misplaced_ratio":0.0092115528992654308,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":89683,"recovering_keys_per_sec":0,"num_objects_recovered":464,"num_bytes_recovered":1077248,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":355},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:28.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:28.307 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:29.043 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:29.045 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:29.097 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-15T21:22:28.413670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":550945428,"num_objects":134512,"num_object_clones":0,"num_object_copies":269024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29911,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":134512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133271,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8150,"num_bytes_recovered":30998547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160558,"ondisk_log_size":160558,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1979364,"kb_used_data":1140212,"kb_used_omap":12,"kb_used_meta":839091,"kb_avail":281136156,"statfs":{"total":289910292480,"available":287883423744,"internally_reserved":0,"allocated":1167577088,"data_stored":1166677452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":859229194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,2,5,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23007232,"num_objects":-5617,"num_object_clones":0,"num_object_copies":-11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":567,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011651"},"pg_stats":[{"pgid":"2.f","version":"48'10137","reported_seq":10388,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128548+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:28.128548+0000","last_peered":"2024-09-15T21:22:28.128548+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:28.128548+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10137,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5093},{"shards":"0,2","objects":2486}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10133","reported_seq":10234,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.098823+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:24.098823+0000","last_peered":"2024-09-15T21:22:24.098823+0000","last_clean":"2024-09-15T21:22:24.098823+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:24.098823+0000","last_undegraded":"2024-09-15T21:22:24.098823+0000","last_fullsized":"2024-09-15T21:22:24.098823+0000","mapping_epoch":36,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8859,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10300","reported_seq":17220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.124306+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:28.124306+0000","last_peered":"2024-09-15T21:22:28.124306+0000","last_clean":"2024-09-15T21:22:28.124306+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:28.124306+0000","last_undegraded":"2024-09-15T21:22:28.124306+0000","last_fullsized":"2024-09-15T21:22:28.124306+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098648,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9023,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10089","reported_seq":10163,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:24.347707+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:24.347707+0000","last_peered":"2024-09-15T21:22:24.347707+0000","last_clean":"2024-09-15T21:22:24.347707+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:24.347707+0000","last_undegraded":"2024-09-15T21:22:24.347707+0000","last_fullsized":"2024-09-15T21:22:24.347707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34738176,"num_objects":8481,"num_object_clones":0,"num_object_copies":16962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10089,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10234","reported_seq":10357,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.144580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:28.144580+0000","last_peered":"2024-09-15T21:22:28.144580+0000","last_clean":"2024-09-15T21:22:28.144580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:28.144580+0000","last_undegraded":"2024-09-15T21:22:28.144580+0000","last_fullsized":"2024-09-15T21:22:28.144580+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8092,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10085","reported_seq":10381,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.087721+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:28.087721+0000","last_peered":"2024-09-15T21:22:28.087721+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:28.087721+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7510,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":2479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10312","reported_seq":10555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.051075+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:28.051075+0000","last_peered":"2024-09-15T21:22:28.051075+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:28.051075+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35192832,"num_objects":8592,"num_object_clones":0,"num_object_copies":17184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7948,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10096","reported_seq":10220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.090442+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:28.090442+0000","last_peered":"2024-09-15T21:22:28.090442+0000","last_clean":"2024-09-15T21:22:28.090442+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:28.090442+0000","last_undegraded":"2024-09-15T21:22:28.090442+0000","last_fullsized":"2024-09-15T21:22:28.090442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'9959","reported_seq":11513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.349218+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:24.349218+0000","last_peered":"2024-09-15T21:22:24.349218+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:24.349218+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":9959,"log_dups_size":0,"ondisk_log_size":9959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8545,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":2891776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2503},{"shards":"2","objects":5790}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9908","reported_seq":10171,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.146515+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:28.146515+0000","last_peered":"2024-09-15T21:22:28.146515+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:28.146515+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7434,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4853},{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'9910","reported_seq":10114,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.047875+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:24.047875+0000","last_peered":"2024-09-15T21:22:24.047875+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:24.047875+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8461,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2501},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10077","reported_seq":10262,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:24.298753+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:24.298753+0000","last_peered":"2024-09-15T21:22:24.298753+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:24.298753+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34697216,"num_objects":8471,"num_object_clones":0,"num_object_copies":16942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2550},{"shards":"2","objects":5154}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10244","reported_seq":10538,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.034821+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:28.034821+0000","last_peered":"2024-09-15T21:22:28.034821+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:28.034821+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34676736,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":2545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10100","reported_seq":10334,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128638+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:28.128638+0000","last_peered":"2024-09-15T21:22:28.128638+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:28.128638+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7685,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4983},{"shards":"0,2","objects":2489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10028","reported_seq":10296,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.020891+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:28.020891+0000","last_peered":"2024-09-15T21:22:28.020891+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:28.020891+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34201600,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10243","reported_seq":17332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.111602+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:28.111602+0000","last_peered":"2024-09-15T21:22:28.111602+0000","last_clean":"2024-09-15T21:22:28.111602+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:28.111602+0000","last_undegraded":"2024-09-15T21:22:28.111602+0000","last_fullsized":"2024-09-15T21:22:28.111602+0000","mapping_epoch":47,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304617,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":550945409,"num_objects":134510,"num_object_clones":0,"num_object_copies":269020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29911,"num_objects_misplaced":2503,"num_objects_unfound":0,"num_objects_dirty":134510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133269,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8149,"num_bytes_recovered":30998528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1165266944,"data_stored":1165185741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160555,"ondisk_log_size":160555,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953534,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625480,"kb_used_data":381368,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93746360,"statfs":{"total":96636764160,"available":95996272640,"internally_reserved":0,"allocated":390520832,"data_stored":390240470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788852,"kb_used_data":455972,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93582988,"statfs":{"total":96636764160,"available":95828979712,"internally_reserved":0,"allocated":466915328,"data_stored":466639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":340848305},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565032,"kb_used_data":302872,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806808,"statfs":{"total":96636764160,"available":96058171392,"internally_reserved":0,"allocated":310140928,"data_stored":309797944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466141184,"data_stored":466137705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309657600,"data_stored":309587427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":389468160,"data_stored":389460609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:32.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:32.905 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:32.906 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:32.920 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-15T21:22:32.414633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548180628,"num_objects":133837,"num_object_clones":0,"num_object_copies":267674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29899,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":133837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133946,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8298,"num_bytes_recovered":31555603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1979516,"kb_used_data":1136204,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":281136004,"statfs":{"total":289910292480,"available":287883268096,"internally_reserved":0,"allocated":1163472896,"data_stored":1162573260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":863489034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17043456,"num_objects":-4161,"num_object_clones":0,"num_object_copies":-8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-194,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4161,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006378"},"pg_stats":[{"pgid":"2.f","version":"48'10137","reported_seq":10388,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128548+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:28.128548+0000","last_peered":"2024-09-15T21:22:28.128548+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:28.128548+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10137,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5093},{"shards":"0,2","objects":2486}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10255","reported_seq":10356,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:29.158713+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:29.158713+0000","last_peered":"2024-09-15T21:22:29.158713+0000","last_clean":"2024-09-15T21:22:29.158713+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:29.158713+0000","last_undegraded":"2024-09-15T21:22:29.158713+0000","last_fullsized":"2024-09-15T21:22:29.158713+0000","mapping_epoch":36,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34344960,"num_objects":8385,"num_object_clones":0,"num_object_copies":16770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10300","reported_seq":17220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.124306+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:28.124306+0000","last_peered":"2024-09-15T21:22:28.124306+0000","last_clean":"2024-09-15T21:22:28.124306+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:28.124306+0000","last_undegraded":"2024-09-15T21:22:28.124306+0000","last_fullsized":"2024-09-15T21:22:28.124306+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098648,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9023,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10246","reported_seq":10320,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:29.328434+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:29.328434+0000","last_peered":"2024-09-15T21:22:29.328434+0000","last_clean":"2024-09-15T21:22:29.328434+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:29.328434+0000","last_undegraded":"2024-09-15T21:22:29.328434+0000","last_fullsized":"2024-09-15T21:22:29.328434+0000","mapping_epoch":36,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10246,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10234","reported_seq":10357,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.144580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:28.144580+0000","last_peered":"2024-09-15T21:22:28.144580+0000","last_clean":"2024-09-15T21:22:28.144580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:28.144580+0000","last_undegraded":"2024-09-15T21:22:28.144580+0000","last_fullsized":"2024-09-15T21:22:28.144580+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8092,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10085","reported_seq":10381,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.087721+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:28.087721+0000","last_peered":"2024-09-15T21:22:28.087721+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:28.087721+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7510,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":2479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10312","reported_seq":10555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.051075+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:28.051075+0000","last_peered":"2024-09-15T21:22:28.051075+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:28.051075+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35192832,"num_objects":8592,"num_object_clones":0,"num_object_copies":17184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7948,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10096","reported_seq":10220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.090442+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:28.090442+0000","last_peered":"2024-09-15T21:22:28.090442+0000","last_clean":"2024-09-15T21:22:28.090442+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:28.090442+0000","last_undegraded":"2024-09-15T21:22:28.090442+0000","last_fullsized":"2024-09-15T21:22:28.090442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10086","reported_seq":11912,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.345842+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:29.345842+0000","last_peered":"2024-09-15T21:22:29.345842+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:29.345842+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33951744,"num_objects":8289,"num_object_clones":0,"num_object_copies":16578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":8289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":3448832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2367},{"shards":"2","objects":5799}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9908","reported_seq":10171,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.146515+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:28.146515+0000","last_peered":"2024-09-15T21:22:28.146515+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:28.146515+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7434,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4853},{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10045","reported_seq":10261,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.321728+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:29.321728+0000","last_peered":"2024-09-15T21:22:29.321728+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:29.321728+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33751040,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2366},{"shards":"2","objects":5025}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10211","reported_seq":10408,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.349063+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:29.349063+0000","last_peered":"2024-09-15T21:22:29.349063+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:29.349063+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8610,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2416},{"shards":"2","objects":5026}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10244","reported_seq":10538,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.034821+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:28.034821+0000","last_peered":"2024-09-15T21:22:28.034821+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:28.034821+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34676736,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":2545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10100","reported_seq":10334,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128638+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:28.128638+0000","last_peered":"2024-09-15T21:22:28.128638+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:28.128638+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7685,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4983},{"shards":"0,2","objects":2489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10028","reported_seq":10296,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.020891+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:28.020891+0000","last_peered":"2024-09-15T21:22:28.020891+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:28.020891+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34201600,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10243","reported_seq":17332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.111602+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:28.111602+0000","last_peered":"2024-09-15T21:22:28.111602+0000","last_clean":"2024-09-15T21:22:28.111602+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:28.111602+0000","last_undegraded":"2024-09-15T21:22:28.111602+0000","last_fullsized":"2024-09-15T21:22:28.111602+0000","mapping_epoch":47,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304617,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548180609,"num_objects":133835,"num_object_clones":0,"num_object_copies":267670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29899,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":133835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133944,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8297,"num_bytes_recovered":31555584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160912896,"data_stored":1160831693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953535,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625632,"kb_used_data":377360,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93746208,"statfs":{"total":96636764160,"available":95996116992,"internally_reserved":0,"allocated":386416640,"data_stored":386136278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788852,"kb_used_data":455972,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93582988,"statfs":{"total":96636764160,"available":95828979712,"internally_reserved":0,"allocated":466915328,"data_stored":466639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":340848305},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565032,"kb_used_data":302872,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806808,"statfs":{"total":96636764160,"available":96058171392,"internally_reserved":0,"allocated":310140928,"data_stored":309797944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466141184,"data_stored":466137705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309657600,"data_stored":309587427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385114112,"data_stored":385106561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:32.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:33.650 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:33.717 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-15T21:22:32.414633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548180628,"num_objects":133837,"num_object_clones":0,"num_object_copies":267674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29899,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":133837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133946,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8298,"num_bytes_recovered":31555603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1979516,"kb_used_data":1136204,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":281136004,"statfs":{"total":289910292480,"available":287883268096,"internally_reserved":0,"allocated":1163472896,"data_stored":1162573260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":863489034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17043456,"num_objects":-4161,"num_object_clones":0,"num_object_copies":-8322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-194,"num_objects_misplaced":-272,"num_objects_unfound":0,"num_objects_dirty":-4161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4161,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":466,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006378"},"pg_stats":[{"pgid":"2.f","version":"48'10137","reported_seq":10388,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128548+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:28.128548+0000","last_peered":"2024-09-15T21:22:28.128548+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:28.128548+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304000,"num_objects":8375,"num_object_clones":0,"num_object_copies":16750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10137,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5093},{"shards":"0,2","objects":2486}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10255","reported_seq":10356,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:29.158713+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:29.158713+0000","last_peered":"2024-09-15T21:22:29.158713+0000","last_clean":"2024-09-15T21:22:29.158713+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:29.158713+0000","last_undegraded":"2024-09-15T21:22:29.158713+0000","last_fullsized":"2024-09-15T21:22:29.158713+0000","mapping_epoch":36,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34344960,"num_objects":8385,"num_object_clones":0,"num_object_copies":16770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10300","reported_seq":17220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.124306+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:28.124306+0000","last_peered":"2024-09-15T21:22:28.124306+0000","last_clean":"2024-09-15T21:22:28.124306+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:28.124306+0000","last_undegraded":"2024-09-15T21:22:28.124306+0000","last_fullsized":"2024-09-15T21:22:28.124306+0000","mapping_epoch":45,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098648,"num_objects":8570,"num_object_clones":0,"num_object_copies":17140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9023,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10246","reported_seq":10320,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:29.328434+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:29.328434+0000","last_peered":"2024-09-15T21:22:29.328434+0000","last_clean":"2024-09-15T21:22:29.328434+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:29.328434+0000","last_undegraded":"2024-09-15T21:22:29.328434+0000","last_fullsized":"2024-09-15T21:22:29.328434+0000","mapping_epoch":36,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10246,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10234","reported_seq":10357,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.144580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:28.144580+0000","last_peered":"2024-09-15T21:22:28.144580+0000","last_clean":"2024-09-15T21:22:28.144580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:28.144580+0000","last_undegraded":"2024-09-15T21:22:28.144580+0000","last_fullsized":"2024-09-15T21:22:28.144580+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8092,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10085","reported_seq":10381,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.087721+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:28.087721+0000","last_peered":"2024-09-15T21:22:28.087721+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:28.087721+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33980416,"num_objects":8296,"num_object_clones":0,"num_object_copies":16592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7510,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":2479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10312","reported_seq":10555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.051075+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:28.051075+0000","last_peered":"2024-09-15T21:22:28.051075+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:28.051075+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35192832,"num_objects":8592,"num_object_clones":0,"num_object_copies":17184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7948,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10096","reported_seq":10220,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.090442+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:28.090442+0000","last_peered":"2024-09-15T21:22:28.090442+0000","last_clean":"2024-09-15T21:22:28.090442+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:28.090442+0000","last_undegraded":"2024-09-15T21:22:28.090442+0000","last_fullsized":"2024-09-15T21:22:28.090442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34029568,"num_objects":8308,"num_object_clones":0,"num_object_copies":16616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7881,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10086","reported_seq":11912,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.345842+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:29.345842+0000","last_peered":"2024-09-15T21:22:29.345842+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:29.345842+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33951744,"num_objects":8289,"num_object_clones":0,"num_object_copies":16578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":8289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":848,"num_bytes_recovered":3448832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2367},{"shards":"2","objects":5799}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'9908","reported_seq":10171,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.146515+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:28.146515+0000","last_peered":"2024-09-15T21:22:28.146515+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:28.146515+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7434,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4853},{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10045","reported_seq":10261,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.321728+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:29.321728+0000","last_peered":"2024-09-15T21:22:29.321728+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:29.321728+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33751040,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2366},{"shards":"2","objects":5025}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10211","reported_seq":10408,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:29.349063+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:29.349063+0000","last_peered":"2024-09-15T21:22:29.349063+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:29.349063+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8610,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2416},{"shards":"2","objects":5026}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10244","reported_seq":10538,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.034821+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:28.034821+0000","last_peered":"2024-09-15T21:22:28.034821+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:28.034821+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34676736,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5114},{"shards":"0,2","objects":2545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10100","reported_seq":10334,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.128638+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:28.128638+0000","last_peered":"2024-09-15T21:22:28.128638+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:28.128638+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7685,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4983},{"shards":"0,2","objects":2489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10028","reported_seq":10296,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.020891+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:28.020891+0000","last_peered":"2024-09-15T21:22:28.020891+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:28.020891+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34201600,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5055},{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10243","reported_seq":17332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.111602+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:28.111602+0000","last_peered":"2024-09-15T21:22:28.111602+0000","last_clean":"2024-09-15T21:22:28.111602+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:28.111602+0000","last_undegraded":"2024-09-15T21:22:28.111602+0000","last_fullsized":"2024-09-15T21:22:28.111602+0000","mapping_epoch":47,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34304617,"num_objects":8376,"num_object_clones":0,"num_object_copies":16752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548180609,"num_objects":133835,"num_object_clones":0,"num_object_copies":267670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29899,"num_objects_misplaced":2367,"num_objects_unfound":0,"num_objects_dirty":133835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133944,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8297,"num_bytes_recovered":31555584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160912896,"data_stored":1160831693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953535,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625632,"kb_used_data":377360,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93746208,"statfs":{"total":96636764160,"available":95996116992,"internally_reserved":0,"allocated":386416640,"data_stored":386136278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149762,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788852,"kb_used_data":455972,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93582988,"statfs":{"total":96636764160,"available":95828979712,"internally_reserved":0,"allocated":466915328,"data_stored":466639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":340848305},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565032,"kb_used_data":302872,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806808,"statfs":{"total":96636764160,"available":96058171392,"internally_reserved":0,"allocated":310140928,"data_stored":309797944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466141184,"data_stored":466137705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309657600,"data_stored":309587427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385114112,"data_stored":385106561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10137", 'reported_seq': 10388, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.128548+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:28.128548+0000', 'last_peered': '2024-09-15T21:22:28.128548+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:28.128548+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34304000, 'num_objects': 8375, 'num_object_clones': 0, 'num_object_copies': 16750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3282, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10137, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 85, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5093}, {'shards': '0,2', 'objects': 2486}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10085", 'reported_seq': 10381, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.087721+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:28.087721+0000', 'last_peered': '2024-09-15T21:22:28.087721+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:28.087721+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33980416, 'num_objects': 8296, 'num_object_clones': 0, 'num_object_copies': 16592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3292, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7510, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5006}, {'shards': '0,2', 'objects': 2479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10312", 'reported_seq': 10555, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.051075+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:28.051075+0000', 'last_peered': '2024-09-15T21:22:28.051075+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:28.051075+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35192832, 'num_objects': 8592, 'num_object_clones': 0, 'num_object_copies': 17184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3415, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7948, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5179}, {'shards': '0,2', 'objects': 2612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10086", 'reported_seq': 11912, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:29.345842+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:29.345842+0000', 'last_peered': '2024-09-15T21:22:29.345842+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:29.345842+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33951744, 'num_objects': 8289, 'num_object_clones': 0, 'num_object_copies': 16578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 2367, 'num_objects_unfound': 0, 'num_objects_dirty': 8289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8672, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 848, 'num_bytes_recovered': 3448832, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2367}, {'shards': '2', 'objects': 5799}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'9908", 'reported_seq': 10171, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.146515+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:28.146515+0000', 'last_peered': '2024-09-15T21:22:28.146515+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:28.146515+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 9908, 'log_dups_size': 0, 'ondisk_log_size': 9908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33284096, 'num_objects': 8126, 'num_object_clones': 0, 'num_object_copies': 16252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3275, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7434, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4853}, {'shards': '0,2', 'objects': 2454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10045", 'reported_seq': 10261, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:29.321728+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:29.321728+0000', 'last_peered': '2024-09-15T21:22:29.321728+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:29.321728+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33751040, 'num_objects': 8240, 'num_object_clones': 0, 'num_object_copies': 16480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3217, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8596, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2366}, {'shards': '2', 'objects': 5025}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10211", 'reported_seq': 10408, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:29.349063+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:29.349063+0000', 'last_peered': '2024-09-15T21:22:29.349063+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:29.349063+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34148352, 'num_objects': 8337, 'num_object_clones': 0, 'num_object_copies': 16674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3313, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8610, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2416}, {'shards': '2', 'objects': 5026}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10244", 'reported_seq': 10538, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.034821+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:28.034821+0000', 'last_peered': '2024-09-15T21:22:28.034821+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:28.034821+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34676736, 'num_objects': 8466, 'num_object_clones': 0, 'num_object_copies': 16932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3353, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7551, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 81, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5114}, {'shards': '0,2', 'objects': 2545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:33.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10100", 'reported_seq': 10334, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.128638+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:28.128638+0000', 'last_peered': '2024-09-15T21:22:28.128638+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:28.128638+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34045952, 'num_objects': 8312, 'num_object_clones': 0, 'num_object_copies': 16624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3332, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7685, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 50, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4983}, {'shards': '0,2', 'objects': 2489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:33.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10028", 'reported_seq': 10296, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.020891+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:28.020891+0000', 'last_peered': '2024-09-15T21:22:28.020891+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:28.020891+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34201600, 'num_objects': 8350, 'num_object_clones': 0, 'num_object_copies': 16700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3297, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7675, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 70, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5055}, {'shards': '0,2', 'objects': 2528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:33.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:34.588 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:34.588 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29840/263502 objects degraded (11.324%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":367,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131751,"data_bytes":539636372,"bytes_used":2014482432,"bytes_avail":287895810048,"bytes_total":289910292480,"degraded_objects":29840,"degraded_total":263502,"degraded_ratio":0.11324392224726947,"misplaced_objects":2206,"misplaced_total":263502,"misplaced_ratio":0.0083718529650628846,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":147764,"recovering_keys_per_sec":0,"num_objects_recovered":686,"num_bytes_recovered":1773568,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":520},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:34.644 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:34.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:35.277 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:35.294 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-15T21:22:34.415075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539636372,"num_objects":131751,"num_object_clones":0,"num_object_copies":263502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29840,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":131751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136032,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8518,"num_bytes_recovered":32215059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967268,"kb_used_data":1123956,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":281148252,"statfs":{"total":289910292480,"available":287895810048,"internally_reserved":0,"allocated":1150930944,"data_stored":1150031308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":863489034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,4,16,6,6,0,0,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25587712,"num_objects":-6247,"num_object_clones":0,"num_object_copies":-12494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":-433,"num_objects_unfound":0,"num_objects_dirty":-6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":1773568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002666"},"pg_stats":[{"pgid":"2.f","version":"48'10274","reported_seq":10539,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.186549+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:33.186549+0000","last_peered":"2024-09-15T21:22:33.186549+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:33.186549+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33742848,"num_objects":8238,"num_object_clones":0,"num_object_copies":16476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10274,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4963},{"shards":"0,2","objects":2349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10390","reported_seq":10491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.306694+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:34.306694+0000","last_peered":"2024-09-15T21:22:34.306694+0000","last_clean":"2024-09-15T21:22:34.306694+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:34.306694+0000","last_undegraded":"2024-09-15T21:22:34.306694+0000","last_fullsized":"2024-09-15T21:22:34.306694+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10443","reported_seq":17363,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.232846+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:33.232846+0000","last_peered":"2024-09-15T21:22:33.232846+0000","last_clean":"2024-09-15T21:22:33.232846+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:33.232846+0000","last_undegraded":"2024-09-15T21:22:33.232846+0000","last_fullsized":"2024-09-15T21:22:33.232846+0000","mapping_epoch":45,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34512920,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9166,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10362","reported_seq":10436,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.346012+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:34.346012+0000","last_peered":"2024-09-15T21:22:34.346012+0000","last_clean":"2024-09-15T21:22:34.346012+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:34.346012+0000","last_undegraded":"2024-09-15T21:22:34.346012+0000","last_fullsized":"2024-09-15T21:22:34.346012+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10362,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10372","reported_seq":10495,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.188663+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:33.188663+0000","last_peered":"2024-09-15T21:22:33.188663+0000","last_clean":"2024-09-15T21:22:33.188663+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:33.188663+0000","last_undegraded":"2024-09-15T21:22:33.188663+0000","last_fullsized":"2024-09-15T21:22:33.188663+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34205696,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10212","reported_seq":10518,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.061301+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:33.061301+0000","last_peered":"2024-09-15T21:22:33.061301+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:33.061301+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7637,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4884},{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10450","reported_seq":10711,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:32.891987+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:32.891987+0000","last_peered":"2024-09-15T21:22:32.891987+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:32.891987+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34627584,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8086,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5050},{"shards":"0,2","objects":2474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10231","reported_seq":10355,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.145349+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:33.145349+0000","last_peered":"2024-09-15T21:22:33.145349+0000","last_clean":"2024-09-15T21:22:33.145349+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:33.145349+0000","last_undegraded":"2024-09-15T21:22:33.145349+0000","last_fullsized":"2024-09-15T21:22:33.145349+0000","mapping_epoch":35,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33476608,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548007+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548007+0000","last_peered":"2024-09-15T21:22:00.548007+0000","last_clean":"2024-09-15T21:22:00.548007+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548007+0000","last_undegraded":"2024-09-15T21:22:00.548007+0000","last_fullsized":"2024-09-15T21:22:00.548007+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10195","reported_seq":12343,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.373375+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:34.373375+0000","last_peered":"2024-09-15T21:22:34.373375+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:34.373375+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8781,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4108288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2206},{"shards":"2","objects":5851}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10022","reported_seq":10291,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.173023+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:33.173023+0000","last_peered":"2024-09-15T21:22:33.173023+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:33.173023+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":2340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10162","reported_seq":10386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.368491+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:34.368491+0000","last_peered":"2024-09-15T21:22:34.368491+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:34.368491+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2249},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10330","reported_seq":10537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.349460+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:34.349460+0000","last_peered":"2024-09-15T21:22:34.349460+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:34.349460+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33660928,"num_objects":8218,"num_object_clones":0,"num_object_copies":16436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8729,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10375","reported_seq":10673,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.215509+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:33.215509+0000","last_peered":"2024-09-15T21:22:33.215509+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:33.215509+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4985},{"shards":"0,2","objects":2414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10237","reported_seq":10501,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.247499+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:33.247499+0000","last_peered":"2024-09-15T21:22:33.247499+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:33.247499+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7822,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4861},{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10164","reported_seq":10449,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.234059+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:33.234059+0000","last_peered":"2024-09-15T21:22:33.234059+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:33.234059+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7811,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4928},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10397","reported_seq":17486,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.058696+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:33.058696+0000","last_peered":"2024-09-15T21:22:33.058696+0000","last_clean":"2024-09-15T21:22:33.058696+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:33.058696+0000","last_undegraded":"2024-09-15T21:22:33.058696+0000","last_fullsized":"2024-09-15T21:22:33.058696+0000","mapping_epoch":47,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33673833,"num_objects":8222,"num_object_clones":0,"num_object_copies":16444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8057,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539636353,"num_objects":131749,"num_object_clones":0,"num_object_copies":263498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29840,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":131749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136030,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8517,"num_bytes_recovered":32215040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1149177856,"data_stored":1149096653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625000,"kb_used_data":372568,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93746840,"statfs":{"total":96636764160,"available":95996764160,"internally_reserved":0,"allocated":381509632,"data_stored":381229270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,1,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769960,"kb_used_data":449560,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93601880,"statfs":{"total":96636764160,"available":95848325120,"internally_reserved":0,"allocated":460349440,"data_stored":460073150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":328068785},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,11,0,2,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572308,"kb_used_data":301828,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93799532,"statfs":{"total":96636764160,"available":96050720768,"internally_reserved":0,"allocated":309071872,"data_stored":308728888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459513856,"data_stored":459510377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308613120,"data_stored":308542947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381050880,"data_stored":381043329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:38.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:38.995 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:38.995 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:39.060 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-15T21:22:38.416014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533987988,"num_objects":130372,"num_object_clones":0,"num_object_copies":260744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29782,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":130372,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137411,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8576,"num_bytes_recovered":32215059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1986624,"kb_used_data":1116936,"kb_used_omap":12,"kb_used_meta":869619,"kb_avail":281128896,"statfs":{"total":289910292480,"available":287875989504,"internally_reserved":0,"allocated":1143742464,"data_stored":1142842828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":890489866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,7,8,1,4,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22487040,"num_objects":-5490,"num_object_clones":0,"num_object_copies":-10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-5490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5490,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":529,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"2.f","version":"48'10385","reported_seq":10666,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.052426+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:38.052426+0000","last_peered":"2024-09-15T21:22:38.052426+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:38.052426+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4860},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10390","reported_seq":10491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.306694+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:34.306694+0000","last_peered":"2024-09-15T21:22:34.306694+0000","last_clean":"2024-09-15T21:22:34.306694+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:34.306694+0000","last_undegraded":"2024-09-15T21:22:34.306694+0000","last_fullsized":"2024-09-15T21:22:34.306694+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10568","reported_seq":17488,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.024971+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:38.024971+0000","last_peered":"2024-09-15T21:22:38.024971+0000","last_clean":"2024-09-15T21:22:38.024971+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:38.024971+0000","last_undegraded":"2024-09-15T21:22:38.024971+0000","last_fullsized":"2024-09-15T21:22:38.024971+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34000920,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9291,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10362","reported_seq":10436,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.346012+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:34.346012+0000","last_peered":"2024-09-15T21:22:34.346012+0000","last_clean":"2024-09-15T21:22:34.346012+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:34.346012+0000","last_undegraded":"2024-09-15T21:22:34.346012+0000","last_fullsized":"2024-09-15T21:22:34.346012+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10362,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10511","reported_seq":10634,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.037145+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:38.037145+0000","last_peered":"2024-09-15T21:22:38.037145+0000","last_clean":"2024-09-15T21:22:38.037145+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:38.037145+0000","last_undegraded":"2024-09-15T21:22:38.037145+0000","last_fullsized":"2024-09-15T21:22:38.037145+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10330","reported_seq":10656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.010282+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:38.010282+0000","last_peered":"2024-09-15T21:22:38.010282+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:38.010282+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976896,"num_objects":8051,"num_object_clones":0,"num_object_copies":16102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7755,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10577","reported_seq":10852,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.038286+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:38.038286+0000","last_peered":"2024-09-15T21:22:38.038286+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:38.038286+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107392,"num_objects":8327,"num_object_clones":0,"num_object_copies":16654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8213,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4930},{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10362","reported_seq":10486,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.040678+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:38.040678+0000","last_peered":"2024-09-15T21:22:38.040678+0000","last_clean":"2024-09-15T21:22:38.040678+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:38.040678+0000","last_undegraded":"2024-09-15T21:22:38.040678+0000","last_fullsized":"2024-09-15T21:22:38.040678+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10195","reported_seq":12343,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.373375+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:34.373375+0000","last_peered":"2024-09-15T21:22:34.373375+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:34.373375+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8781,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4108288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2206},{"shards":"2","objects":5851}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10145","reported_seq":10432,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.293470+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:38.293470+0000","last_peered":"2024-09-15T21:22:38.293470+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:38.293470+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4628},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10162","reported_seq":10386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.368491+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:34.368491+0000","last_peered":"2024-09-15T21:22:34.368491+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:34.368491+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2249},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10330","reported_seq":10537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.349460+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:34.349460+0000","last_peered":"2024-09-15T21:22:34.349460+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:34.349460+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33660928,"num_objects":8218,"num_object_clones":0,"num_object_copies":16436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8729,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10502","reported_seq":10818,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.078370+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:38.078370+0000","last_peered":"2024-09-15T21:22:38.078370+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:38.078370+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7809,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4867},{"shards":"0,2","objects":2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10367","reported_seq":10655,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.290038+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:38.290038+0000","last_peered":"2024-09-15T21:22:38.290038+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:38.290038+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952320,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7952,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4743},{"shards":"0,2","objects":2222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10289","reported_seq":10581,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:37.958045+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:37.958045+0000","last_peered":"2024-09-15T21:22:37.958045+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:37.958045+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7936,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4806},{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10520","reported_seq":17609,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:37.957907+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:37.957907+0000","last_peered":"2024-09-15T21:22:37.957907+0000","last_clean":"2024-09-15T21:22:37.957907+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:37.957907+0000","last_undegraded":"2024-09-15T21:22:37.957907+0000","last_fullsized":"2024-09-15T21:22:37.957907+0000","mapping_epoch":47,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33170025,"num_objects":8099,"num_object_clones":0,"num_object_copies":16198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533987969,"num_objects":130370,"num_object_clones":0,"num_object_copies":260740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29782,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":130370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137409,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8575,"num_bytes_recovered":32215040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142304768,"data_stored":1142223565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625000,"kb_used_data":372568,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93746840,"statfs":{"total":96636764160,"available":95996764160,"internally_reserved":0,"allocated":381509632,"data_stored":381229270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,1,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786032,"kb_used_data":443416,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93585808,"statfs":{"total":96636764160,"available":95831867392,"internally_reserved":0,"allocated":454057984,"data_stored":453781694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,0,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575592,"kb_used_data":300952,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93796248,"statfs":{"total":96636764160,"available":96047357952,"internally_reserved":0,"allocated":308174848,"data_stored":307831864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,3,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453443584,"data_stored":453440105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307810304,"data_stored":307740131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381050880,"data_stored":381043329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:39.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:39.778 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:39.780 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:39.824 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-15T21:22:38.416014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533987988,"num_objects":130372,"num_object_clones":0,"num_object_copies":260744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29782,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":130372,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137411,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8576,"num_bytes_recovered":32215059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1986624,"kb_used_data":1116936,"kb_used_omap":12,"kb_used_meta":869619,"kb_avail":281128896,"statfs":{"total":289910292480,"available":287875989504,"internally_reserved":0,"allocated":1143742464,"data_stored":1142842828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":890489866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,7,8,1,4,0,0,7],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22487040,"num_objects":-5490,"num_object_clones":0,"num_object_copies":-10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-5490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5490,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":529,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"2.f","version":"48'10385","reported_seq":10666,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.052426+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:38.052426+0000","last_peered":"2024-09-15T21:22:38.052426+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:38.052426+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4860},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10390","reported_seq":10491,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.306694+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:34.306694+0000","last_peered":"2024-09-15T21:22:34.306694+0000","last_clean":"2024-09-15T21:22:34.306694+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:34.306694+0000","last_undegraded":"2024-09-15T21:22:34.306694+0000","last_fullsized":"2024-09-15T21:22:34.306694+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10568","reported_seq":17488,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.024971+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:38.024971+0000","last_peered":"2024-09-15T21:22:38.024971+0000","last_clean":"2024-09-15T21:22:38.024971+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:38.024971+0000","last_undegraded":"2024-09-15T21:22:38.024971+0000","last_fullsized":"2024-09-15T21:22:38.024971+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34000920,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9291,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10362","reported_seq":10436,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:34.346012+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:34.346012+0000","last_peered":"2024-09-15T21:22:34.346012+0000","last_clean":"2024-09-15T21:22:34.346012+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:34.346012+0000","last_undegraded":"2024-09-15T21:22:34.346012+0000","last_fullsized":"2024-09-15T21:22:34.346012+0000","mapping_epoch":36,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10362,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10511","reported_seq":10634,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.037145+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:38.037145+0000","last_peered":"2024-09-15T21:22:38.037145+0000","last_clean":"2024-09-15T21:22:38.037145+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:38.037145+0000","last_undegraded":"2024-09-15T21:22:38.037145+0000","last_fullsized":"2024-09-15T21:22:38.037145+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10330","reported_seq":10656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.010282+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:38.010282+0000","last_peered":"2024-09-15T21:22:38.010282+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:38.010282+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976896,"num_objects":8051,"num_object_clones":0,"num_object_copies":16102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7755,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10577","reported_seq":10852,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.038286+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:38.038286+0000","last_peered":"2024-09-15T21:22:38.038286+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:38.038286+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107392,"num_objects":8327,"num_object_clones":0,"num_object_copies":16654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8213,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4930},{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10362","reported_seq":10486,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.040678+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:38.040678+0000","last_peered":"2024-09-15T21:22:38.040678+0000","last_clean":"2024-09-15T21:22:38.040678+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:38.040678+0000","last_undegraded":"2024-09-15T21:22:38.040678+0000","last_fullsized":"2024-09-15T21:22:38.040678+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10195","reported_seq":12343,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.373375+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:34.373375+0000","last_peered":"2024-09-15T21:22:34.373375+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:34.373375+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8781,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4108288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2206},{"shards":"2","objects":5851}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10145","reported_seq":10432,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.293470+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:38.293470+0000","last_peered":"2024-09-15T21:22:38.293470+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:38.293470+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4628},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10162","reported_seq":10386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.368491+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:34.368491+0000","last_peered":"2024-09-15T21:22:34.368491+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:34.368491+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33271808,"num_objects":8123,"num_object_clones":0,"num_object_copies":16246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8713,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2249},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10330","reported_seq":10537,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:34.349460+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:34.349460+0000","last_peered":"2024-09-15T21:22:34.349460+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:34.349460+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33660928,"num_objects":8218,"num_object_clones":0,"num_object_copies":16436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8729,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2297},{"shards":"2","objects":4912}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10502","reported_seq":10818,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.078370+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:38.078370+0000","last_peered":"2024-09-15T21:22:38.078370+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:38.078370+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7809,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4867},{"shards":"0,2","objects":2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10367","reported_seq":10655,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.290038+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:38.290038+0000","last_peered":"2024-09-15T21:22:38.290038+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:38.290038+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952320,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7952,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4743},{"shards":"0,2","objects":2222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10289","reported_seq":10581,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:37.958045+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:37.958045+0000","last_peered":"2024-09-15T21:22:37.958045+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:37.958045+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7936,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4806},{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10520","reported_seq":17609,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:37.957907+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:37.957907+0000","last_peered":"2024-09-15T21:22:37.957907+0000","last_clean":"2024-09-15T21:22:37.957907+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:37.957907+0000","last_undegraded":"2024-09-15T21:22:37.957907+0000","last_fullsized":"2024-09-15T21:22:37.957907+0000","mapping_epoch":47,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33170025,"num_objects":8099,"num_object_clones":0,"num_object_copies":16198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533987969,"num_objects":130370,"num_object_clones":0,"num_object_copies":260740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29782,"num_objects_misplaced":2206,"num_objects_unfound":0,"num_objects_dirty":130370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137409,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8575,"num_bytes_recovered":32215040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142304768,"data_stored":1142223565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953536,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625000,"kb_used_data":372568,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93746840,"statfs":{"total":96636764160,"available":95996764160,"internally_reserved":0,"allocated":381509632,"data_stored":381229270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,1,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786032,"kb_used_data":443416,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93585808,"statfs":{"total":96636764160,"available":95831867392,"internally_reserved":0,"allocated":454057984,"data_stored":453781694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,0,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575592,"kb_used_data":300952,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93796248,"statfs":{"total":96636764160,"available":96047357952,"internally_reserved":0,"allocated":308174848,"data_stored":307831864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,3,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453443584,"data_stored":453440105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307810304,"data_stored":307740131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381050880,"data_stored":381043329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10385", 'reported_seq': 10666, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.052426+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:38.052426+0000', 'last_peered': '2024-09-15T21:22:38.052426+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:38.052426+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33288192, 'num_objects': 8127, 'num_object_clones': 0, 'num_object_copies': 16254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3267, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10385, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 100, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4860}, {'shards': '0,2', 'objects': 2238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10330", 'reported_seq': 10656, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.010282+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:38.010282+0000', 'last_peered': '2024-09-15T21:22:38.010282+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:38.010282+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32976896, 'num_objects': 8051, 'num_object_clones': 0, 'num_object_copies': 16102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3277, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7755, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 96, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4776}, {'shards': '0,2', 'objects': 2234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10577", 'reported_seq': 10852, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.038286+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:38.038286+0000', 'last_peered': '2024-09-15T21:22:38.038286+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:38.038286+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34107392, 'num_objects': 8327, 'num_object_clones': 0, 'num_object_copies': 16654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3399, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8213, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 74, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4930}, {'shards': '0,2', 'objects': 2347}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:39.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10195", 'reported_seq': 12343, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:34.373375+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:34.373375+0000', 'last_peered': '2024-09-15T21:22:34.373375+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:34.373375+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33505280, 'num_objects': 8180, 'num_object_clones': 0, 'num_object_copies': 16360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 2206, 'num_objects_unfound': 0, 'num_objects_dirty': 8180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8781, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1009, 'num_bytes_recovered': 4108288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2206}, {'shards': '2', 'objects': 5851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10145", 'reported_seq': 10432, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.293470+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:38.293470+0000', 'last_peered': '2024-09-15T21:22:38.293470+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:38.293470+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313344, 'num_objects': 7889, 'num_object_clones': 0, 'num_object_copies': 15778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7671, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 80, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4628}, {'shards': '0,2', 'objects': 2217}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10162", 'reported_seq': 10386, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:34.368491+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:34.368491+0000', 'last_peered': '2024-09-15T21:22:34.368491+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:34.368491+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'100", 'ondisk_log_start': "37'100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33271808, 'num_objects': 8123, 'num_object_clones': 0, 'num_object_copies': 16246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3213, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8713, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2249}, {'shards': '2', 'objects': 4912}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10330", 'reported_seq': 10537, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:34.349460+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:34.349460+0000', 'last_peered': '2024-09-15T21:22:34.349460+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:34.349460+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33660928, 'num_objects': 8218, 'num_object_clones': 0, 'num_object_copies': 16436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3308, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8729, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2297}, {'shards': '2', 'objects': 4912}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10502", 'reported_seq': 10818, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.078370+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:38.078370+0000', 'last_peered': '2024-09-15T21:22:38.078370+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:38.078370+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33619968, 'num_objects': 8208, 'num_object_clones': 0, 'num_object_copies': 16416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3342, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7809, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 92, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4867}, {'shards': '0,2', 'objects': 2287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10367", 'reported_seq': 10655, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.290038+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:38.290038+0000', 'last_peered': '2024-09-15T21:22:38.290038+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:38.290038+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'300", 'ondisk_log_start': "37'300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32952320, 'num_objects': 8045, 'num_object_clones': 0, 'num_object_copies': 16090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7952, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4743}, {'shards': '0,2', 'objects': 2222}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:39.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10289", 'reported_seq': 10581, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:37.958045+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:37.958045+0000', 'last_peered': '2024-09-15T21:22:37.958045+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:37.958045+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33132544, 'num_objects': 8089, 'num_object_clones': 0, 'num_object_copies': 16178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7936, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 82, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4806}, {'shards': '0,2', 'objects': 2267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:39.828 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:40.756 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:40.756 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29768/259514 objects degraded (11.471%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":373,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129757,"data_bytes":531468948,"bytes_used":2035240960,"bytes_avail":287875051520,"bytes_total":289910292480,"degraded_objects":29768,"degraded_total":259514,"degraded_ratio":0.11470672102468459,"misplaced_objects":2032,"misplaced_total":259514,"misplaced_ratio":0.0078300207310588246,"recovering_objects_per_sec":51,"recovering_bytes_per_sec":160654,"recovering_keys_per_sec":0,"num_objects_recovered":616,"num_bytes_recovered":1929216,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:40.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:40.802 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:41.689 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:41.732 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-15T21:22:40.422114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":531468948,"num_objects":129757,"num_object_clones":0,"num_object_copies":259514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29768,"num_objects_misplaced":2032,"num_objects_unfound":0,"num_objects_dirty":129757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138026,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8766,"num_bytes_recovered":32927763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1987540,"kb_used_data":1113692,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":281127980,"statfs":{"total":289910292480,"available":287875051520,"internally_reserved":0,"allocated":1140420608,"data_stored":1139520972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":894749706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,6,8,0,4,0,0,6,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19476480,"num_objects":-4755,"num_object_clones":0,"num_object_copies":-9510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-143,"num_objects_misplaced":-471,"num_objects_unfound":0,"num_objects_dirty":-4755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":616,"num_bytes_recovered":1929216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008444"},"pg_stats":[{"pgid":"2.f","version":"48'10385","reported_seq":10666,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.052426+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:38.052426+0000","last_peered":"2024-09-15T21:22:38.052426+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:38.052426+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33288192,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4860},{"shards":"0,2","objects":2238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10521","reported_seq":10622,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:39.373784+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:39.373784+0000","last_peered":"2024-09-15T21:22:39.373784+0000","last_clean":"2024-09-15T21:22:39.373784+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:39.373784+0000","last_undegraded":"2024-09-15T21:22:39.373784+0000","last_fullsized":"2024-09-15T21:22:39.373784+0000","mapping_epoch":36,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9247,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10568","reported_seq":17488,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.024971+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:38.024971+0000","last_peered":"2024-09-15T21:22:38.024971+0000","last_clean":"2024-09-15T21:22:38.024971+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:38.024971+0000","last_undegraded":"2024-09-15T21:22:38.024971+0000","last_fullsized":"2024-09-15T21:22:38.024971+0000","mapping_epoch":45,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34000920,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9291,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10466","reported_seq":10540,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:39.264720+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:39.264720+0000","last_peered":"2024-09-15T21:22:39.264720+0000","last_clean":"2024-09-15T21:22:39.264720+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:39.264720+0000","last_undegraded":"2024-09-15T21:22:39.264720+0000","last_fullsized":"2024-09-15T21:22:39.264720+0000","mapping_epoch":36,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33193984,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10466,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10511","reported_seq":10634,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.037145+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:38.037145+0000","last_peered":"2024-09-15T21:22:38.037145+0000","last_clean":"2024-09-15T21:22:38.037145+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:38.037145+0000","last_undegraded":"2024-09-15T21:22:38.037145+0000","last_fullsized":"2024-09-15T21:22:38.037145+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33636352,"num_objects":8212,"num_object_clones":0,"num_object_copies":16424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10330","reported_seq":10656,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.010282+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:38.010282+0000","last_peered":"2024-09-15T21:22:38.010282+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:38.010282+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32976896,"num_objects":8051,"num_object_clones":0,"num_object_copies":16102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7755,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":2234}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10577","reported_seq":10852,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.038286+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:38.038286+0000","last_peered":"2024-09-15T21:22:38.038286+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:38.038286+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107392,"num_objects":8327,"num_object_clones":0,"num_object_copies":16654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8213,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4930},{"shards":"0,2","objects":2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10362","reported_seq":10486,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.040678+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:38.040678+0000","last_peered":"2024-09-15T21:22:38.040678+0000","last_clean":"2024-09-15T21:22:38.040678+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:38.040678+0000","last_undegraded":"2024-09-15T21:22:38.040678+0000","last_fullsized":"2024-09-15T21:22:38.040678+0000","mapping_epoch":35,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32940032,"num_objects":8042,"num_object_clones":0,"num_object_copies":16084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10318","reported_seq":12817,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:39.380176+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:39.380176+0000","last_peered":"2024-09-15T21:22:39.380176+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:39.380176+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001472,"num_objects":8057,"num_object_clones":0,"num_object_copies":16114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":2032,"num_objects_unfound":0,"num_objects_dirty":8057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8904,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1185,"num_bytes_recovered":4820992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2032},{"shards":"2","objects":5902}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10145","reported_seq":10432,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.293470+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:38.293470+0000","last_peered":"2024-09-15T21:22:38.293470+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:38.293470+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'100","ondisk_log_start":"37'100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7671,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4628},{"shards":"0,2","objects":2217}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10298","reported_seq":10536,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:39.389536+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:39.389536+0000","last_peered":"2024-09-15T21:22:39.389536+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:39.389536+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8849,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2113},{"shards":"2","objects":4783}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10451","reported_seq":10672,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:39.346640+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:39.346640+0000","last_peered":"2024-09-15T21:22:39.346640+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:39.346640+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33165312,"num_objects":8097,"num_object_clones":0,"num_object_copies":16194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8850,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2176},{"shards":"2","objects":4798}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10502","reported_seq":10818,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.078370+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:38.078370+0000","last_peered":"2024-09-15T21:22:38.078370+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:38.078370+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33619968,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7809,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4867},{"shards":"0,2","objects":2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10367","reported_seq":10655,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.290038+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:38.290038+0000","last_peered":"2024-09-15T21:22:38.290038+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:38.290038+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'300","ondisk_log_start":"37'300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952320,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7952,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4743},{"shards":"0,2","objects":2222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10289","reported_seq":10581,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:37.958045+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:37.958045+0000","last_peered":"2024-09-15T21:22:37.958045+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:37.958045+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7936,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4806},{"shards":"0,2","objects":2267}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10520","reported_seq":17609,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:37.957907+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:37.957907+0000","last_peered":"2024-09-15T21:22:37.957907+0000","last_clean":"2024-09-15T21:22:37.957907+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:37.957907+0000","last_undegraded":"2024-09-15T21:22:37.957907+0000","last_fullsized":"2024-09-15T21:22:37.957907+0000","mapping_epoch":47,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33170025,"num_objects":8099,"num_object_clones":0,"num_object_copies":16198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":531468929,"num_objects":129755,"num_object_clones":0,"num_object_copies":259510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29768,"num_objects_misplaced":2032,"num_objects_unfound":0,"num_objects_dirty":129755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138024,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8765,"num_bytes_recovered":32927744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1138212864,"data_stored":1138131661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953537,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625916,"kb_used_data":369324,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93745924,"statfs":{"total":96636764160,"available":95995826176,"internally_reserved":0,"allocated":378187776,"data_stored":377907414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,0,0,0,0,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149764,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786032,"kb_used_data":443416,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93585808,"statfs":{"total":96636764160,"available":95831867392,"internally_reserved":0,"allocated":454057984,"data_stored":453781694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,0,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575592,"kb_used_data":300952,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93796248,"statfs":{"total":96636764160,"available":96047357952,"internally_reserved":0,"allocated":308174848,"data_stored":307831864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":281210540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,3,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453443584,"data_stored":453440105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307810304,"data_stored":307740131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376958976,"data_stored":376951425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:44.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:45.351 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:45.353 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:45.427 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-15T21:22:44.422996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522580628,"num_objects":127587,"num_object_clones":0,"num_object_copies":255174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127587,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140196,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8977,"num_bytes_recovered":33431571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1963648,"kb_used_data":1102284,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281151872,"statfs":{"total":289910292480,"available":287899516928,"internally_reserved":0,"allocated":1128738816,"data_stored":1127846138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":881970186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,4,22,5,1,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25600000,"num_objects":-6250,"num_object_clones":0,"num_object_copies":-12500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":-458,"num_objects_unfound":0,"num_objects_dirty":-6250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":679,"num_bytes_recovered":1875968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008363"},"pg_stats":[{"pgid":"2.f","version":"48'10532","reported_seq":10847,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.293649+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:43.293649+0000","last_peered":"2024-09-15T21:22:43.293649+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:43.293649+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4730},{"shards":"0,2","objects":2091}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10641","reported_seq":10740,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.407804+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:44.407804+0000","last_peered":"2024-09-15T21:22:44.407804+0000","last_clean":"2024-09-15T21:22:44.407804+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:44.407804+0000","last_undegraded":"2024-09-15T21:22:44.407804+0000","last_fullsized":"2024-09-15T21:22:44.407804+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10710","reported_seq":17630,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.139111+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:43.139111+0000","last_peered":"2024-09-15T21:22:43.139111+0000","last_clean":"2024-09-15T21:22:43.139111+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:43.139111+0000","last_undegraded":"2024-09-15T21:22:43.139111+0000","last_fullsized":"2024-09-15T21:22:43.139111+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419288,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9433,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10609","reported_seq":10683,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.379590+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:44.379590+0000","last_peered":"2024-09-15T21:22:44.379590+0000","last_clean":"2024-09-15T21:22:44.379590+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:44.379590+0000","last_undegraded":"2024-09-15T21:22:44.379590+0000","last_fullsized":"2024-09-15T21:22:44.379590+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10652","reported_seq":10775,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.243479+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:43.243479+0000","last_peered":"2024-09-15T21:22:43.243479+0000","last_clean":"2024-09-15T21:22:43.243479+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:43.243479+0000","last_undegraded":"2024-09-15T21:22:43.243479+0000","last_fullsized":"2024-09-15T21:22:43.243479+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10462","reported_seq":10802,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.183886+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:43.183886+0000","last_peered":"2024-09-15T21:22:43.183886+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:43.183886+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4651},{"shards":"0,2","objects":2102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10716","reported_seq":11024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317908+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:43.317908+0000","last_peered":"2024-09-15T21:22:43.317908+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:43.317908+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8352,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4807},{"shards":"0,2","objects":2208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10499","reported_seq":10623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.249521+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:43.249521+0000","last_peered":"2024-09-15T21:22:43.249521+0000","last_clean":"2024-09-15T21:22:43.249521+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:43.249521+0000","last_undegraded":"2024-09-15T21:22:43.249521+0000","last_fullsized":"2024-09-15T21:22:43.249521+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10470","reported_seq":13215,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415839+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:44.415839+0000","last_peered":"2024-09-15T21:22:44.415839+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:44.415839+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1308,"num_bytes_recovered":5324800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1909},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10275","reported_seq":10587,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317294+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:43.317294+0000","last_peered":"2024-09-15T21:22:43.317294+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:43.317294+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7801,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":2087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10430","reported_seq":10670,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.360454+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:44.360454+0000","last_peered":"2024-09-15T21:22:44.360454+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:44.360454+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174080,"num_objects":7855,"num_object_clones":0,"num_object_copies":15710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1981},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10589","reported_seq":10816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415886+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:44.415886+0000","last_peered":"2024-09-15T21:22:44.415886+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:44.415886+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32600064,"num_objects":7959,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2038},{"shards":"2","objects":4663}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10625","reported_seq":10967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.253926+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:43.253926+0000","last_peered":"2024-09-15T21:22:43.253926+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:43.253926+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33116160,"num_objects":8085,"num_object_clones":0,"num_object_copies":16170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7932,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4757},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10506","reported_seq":10806,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.222903+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:43.222903+0000","last_peered":"2024-09-15T21:22:43.222903+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:43.222903+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32382976,"num_objects":7906,"num_object_clones":0,"num_object_copies":15812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4610},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10430","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.290945+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:43.290945+0000","last_peered":"2024-09-15T21:22:43.290945+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:43.290945+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4676},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10634","reported_seq":17723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.207392+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:43.207392+0000","last_peered":"2024-09-15T21:22:43.207392+0000","last_clean":"2024-09-15T21:22:43.207392+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:43.207392+0000","last_undegraded":"2024-09-15T21:22:43.207392+0000","last_fullsized":"2024-09-15T21:22:43.207392+0000","mapping_epoch":47,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32703081,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8294,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522580609,"num_objects":127585,"num_object_clones":0,"num_object_copies":255170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140194,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8976,"num_bytes_recovered":33431552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126080512,"data_stored":1126006267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608980,"kb_used_data":364868,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93762860,"statfs":{"total":96636764160,"available":96013168640,"internally_reserved":0,"allocated":373624832,"data_stored":373344470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788352,"kb_used_data":437420,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93583488,"statfs":{"total":96636764160,"available":95829491712,"internally_reserved":0,"allocated":447918080,"data_stored":447641790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,16,0,0,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566316,"kb_used_data":299996,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93805524,"statfs":{"total":96636764160,"available":96056856576,"internally_reserved":0,"allocated":307195904,"data_stored":306859878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446955520,"data_stored":446952041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306733056,"data_stored":306669841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372391936,"data_stored":372384385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:45.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:46.161 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:46.181 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-15T21:22:44.422996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522580628,"num_objects":127587,"num_object_clones":0,"num_object_copies":255174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127587,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140196,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8977,"num_bytes_recovered":33431571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1963648,"kb_used_data":1102284,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281151872,"statfs":{"total":289910292480,"available":287899516928,"internally_reserved":0,"allocated":1128738816,"data_stored":1127846138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":881970186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,4,22,5,1,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25600000,"num_objects":-6250,"num_object_clones":0,"num_object_copies":-12500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":-458,"num_objects_unfound":0,"num_objects_dirty":-6250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":679,"num_bytes_recovered":1875968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008363"},"pg_stats":[{"pgid":"2.f","version":"48'10532","reported_seq":10847,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.293649+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:43.293649+0000","last_peered":"2024-09-15T21:22:43.293649+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:43.293649+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4730},{"shards":"0,2","objects":2091}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10641","reported_seq":10740,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.407804+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:44.407804+0000","last_peered":"2024-09-15T21:22:44.407804+0000","last_clean":"2024-09-15T21:22:44.407804+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:44.407804+0000","last_undegraded":"2024-09-15T21:22:44.407804+0000","last_fullsized":"2024-09-15T21:22:44.407804+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10710","reported_seq":17630,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.139111+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:43.139111+0000","last_peered":"2024-09-15T21:22:43.139111+0000","last_clean":"2024-09-15T21:22:43.139111+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:43.139111+0000","last_undegraded":"2024-09-15T21:22:43.139111+0000","last_fullsized":"2024-09-15T21:22:43.139111+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419288,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9433,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10609","reported_seq":10683,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.379590+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:44.379590+0000","last_peered":"2024-09-15T21:22:44.379590+0000","last_clean":"2024-09-15T21:22:44.379590+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:44.379590+0000","last_undegraded":"2024-09-15T21:22:44.379590+0000","last_fullsized":"2024-09-15T21:22:44.379590+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10652","reported_seq":10775,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.243479+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:43.243479+0000","last_peered":"2024-09-15T21:22:43.243479+0000","last_clean":"2024-09-15T21:22:43.243479+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:43.243479+0000","last_undegraded":"2024-09-15T21:22:43.243479+0000","last_fullsized":"2024-09-15T21:22:43.243479+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10462","reported_seq":10802,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.183886+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:43.183886+0000","last_peered":"2024-09-15T21:22:43.183886+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:43.183886+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4651},{"shards":"0,2","objects":2102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10716","reported_seq":11024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317908+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:43.317908+0000","last_peered":"2024-09-15T21:22:43.317908+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:43.317908+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8352,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4807},{"shards":"0,2","objects":2208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10499","reported_seq":10623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.249521+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:43.249521+0000","last_peered":"2024-09-15T21:22:43.249521+0000","last_clean":"2024-09-15T21:22:43.249521+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:43.249521+0000","last_undegraded":"2024-09-15T21:22:43.249521+0000","last_fullsized":"2024-09-15T21:22:43.249521+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10470","reported_seq":13215,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415839+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:44.415839+0000","last_peered":"2024-09-15T21:22:44.415839+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:44.415839+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1308,"num_bytes_recovered":5324800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1909},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10275","reported_seq":10587,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317294+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:43.317294+0000","last_peered":"2024-09-15T21:22:43.317294+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:43.317294+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7801,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":2087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10430","reported_seq":10670,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.360454+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:44.360454+0000","last_peered":"2024-09-15T21:22:44.360454+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:44.360454+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174080,"num_objects":7855,"num_object_clones":0,"num_object_copies":15710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1981},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10589","reported_seq":10816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415886+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:44.415886+0000","last_peered":"2024-09-15T21:22:44.415886+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:44.415886+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32600064,"num_objects":7959,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2038},{"shards":"2","objects":4663}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10625","reported_seq":10967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.253926+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:43.253926+0000","last_peered":"2024-09-15T21:22:43.253926+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:43.253926+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33116160,"num_objects":8085,"num_object_clones":0,"num_object_copies":16170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7932,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4757},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10506","reported_seq":10806,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.222903+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:43.222903+0000","last_peered":"2024-09-15T21:22:43.222903+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:43.222903+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32382976,"num_objects":7906,"num_object_clones":0,"num_object_copies":15812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4610},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10430","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.290945+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:43.290945+0000","last_peered":"2024-09-15T21:22:43.290945+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:43.290945+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4676},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10634","reported_seq":17723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.207392+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:43.207392+0000","last_peered":"2024-09-15T21:22:43.207392+0000","last_clean":"2024-09-15T21:22:43.207392+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:43.207392+0000","last_undegraded":"2024-09-15T21:22:43.207392+0000","last_fullsized":"2024-09-15T21:22:43.207392+0000","mapping_epoch":47,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32703081,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8294,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522580609,"num_objects":127585,"num_object_clones":0,"num_object_copies":255170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140194,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8976,"num_bytes_recovered":33431552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126080512,"data_stored":1126006267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608980,"kb_used_data":364868,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93762860,"statfs":{"total":96636764160,"available":96013168640,"internally_reserved":0,"allocated":373624832,"data_stored":373344470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788352,"kb_used_data":437420,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93583488,"statfs":{"total":96636764160,"available":95829491712,"internally_reserved":0,"allocated":447918080,"data_stored":447641790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,16,0,0,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566316,"kb_used_data":299996,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93805524,"statfs":{"total":96636764160,"available":96056856576,"internally_reserved":0,"allocated":307195904,"data_stored":306859878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446955520,"data_stored":446952041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306733056,"data_stored":306669841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372391936,"data_stored":372384385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:46.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:46.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10532", 'reported_seq': 10847, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.293649+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:43.293649+0000', 'last_peered': '2024-09-15T21:22:43.293649+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:43.293649+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32686080, 'num_objects': 7980, 'num_object_clones': 0, 'num_object_copies': 15960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3250, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7980, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10532, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 117, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4730}, {'shards': '0,2', 'objects': 2091}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10462", 'reported_seq': 10802, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.183886+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:43.183886+0000', 'last_peered': '2024-09-15T21:22:43.183886+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:43.183886+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32436224, 'num_objects': 7919, 'num_object_clones': 0, 'num_object_copies': 15838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7887, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 103, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4651}, {'shards': '0,2', 'objects': 2102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10716", 'reported_seq': 11024, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.317908+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:43.317908+0000', 'last_peered': '2024-09-15T21:22:43.317908+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:43.317908+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33538048, 'num_objects': 8188, 'num_object_clones': 0, 'num_object_copies': 16376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3383, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8352, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 90, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4807}, {'shards': '0,2', 'objects': 2208}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10470", 'reported_seq': 13215, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:44.415839+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:44.415839+0000', 'last_peered': '2024-09-15T21:22:44.415839+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:44.415839+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32378880, 'num_objects': 7905, 'num_object_clones': 0, 'num_object_copies': 15810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 1909, 'num_objects_unfound': 0, 'num_objects_dirty': 7905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9056, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1308, 'num_bytes_recovered': 5324800, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1909}, {'shards': '2', 'objects': 5873}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10275", 'reported_seq': 10587, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.317294+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:43.317294+0000', 'last_peered': '2024-09-15T21:22:43.317294+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:43.317294+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'200", 'ondisk_log_start': "37'200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31780864, 'num_objects': 7759, 'num_object_clones': 0, 'num_object_copies': 15518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3249, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7801, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4512}, {'shards': '0,2', 'objects': 2087}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:46.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10430", 'reported_seq': 10670, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:44.360454+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:44.360454+0000', 'last_peered': '2024-09-15T21:22:44.360454+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:44.360454+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32174080, 'num_objects': 7855, 'num_object_clones': 0, 'num_object_copies': 15710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3205, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8981, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1981}, {'shards': '2', 'objects': 4652}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10589", 'reported_seq': 10816, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:44.415886+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:44.415886+0000', 'last_peered': '2024-09-15T21:22:44.415886+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:44.415886+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32600064, 'num_objects': 7959, 'num_object_clones': 0, 'num_object_copies': 15918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8988, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 54, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2038}, {'shards': '2', 'objects': 4663}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10625", 'reported_seq': 10967, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.253926+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:43.253926+0000', 'last_peered': '2024-09-15T21:22:43.253926+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:43.253926+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33116160, 'num_objects': 8085, 'num_object_clones': 0, 'num_object_copies': 16170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3329, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7932, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 105, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4757}, {'shards': '0,2', 'objects': 2164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10506", 'reported_seq': 10806, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.222903+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:43.222903+0000', 'last_peered': '2024-09-15T21:22:43.222903+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:43.222903+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32382976, 'num_objects': 7906, 'num_object_clones': 0, 'num_object_copies': 15812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3299, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8091, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4610}, {'shards': '0,2', 'objects': 2083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:46.184 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10430", 'reported_seq': 10744, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.290945+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:43.290945+0000', 'last_peered': '2024-09-15T21:22:43.290945+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:43.290945+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32555008, 'num_objects': 7948, 'num_object_clones': 0, 'num_object_copies': 15896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3274, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8077, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 93, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4676}, {'shards': '0,2', 'objects': 2126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:46.184 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:46.725 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:46.725 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29680/255174 objects degraded (11.631%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":379,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":127587,"data_bytes":522580628,"bytes_used":2010775552,"bytes_avail":287899516928,"bytes_total":289910292480,"degraded_objects":29680,"degraded_total":255174,"degraded_ratio":0.11631279048805913,"misplaced_objects":1909,"misplaced_total":255174,"misplaced_ratio":0.0074811697116477383,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":101304,"recovering_keys_per_sec":0,"num_objects_recovered":459,"num_bytes_recovered":1216512,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":346},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:46.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:46.765 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:47.421 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:47.469 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-15T21:22:46.423493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":522580628,"num_objects":127587,"num_object_clones":0,"num_object_copies":255174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127587,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140196,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8977,"num_bytes_recovered":33431571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1963648,"kb_used_data":1102284,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":281151872,"statfs":{"total":289910292480,"available":287899516928,"internally_reserved":0,"allocated":1128738816,"data_stored":1127846138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":881970186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,4,22,5,1,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17055744,"num_objects":-4164,"num_object_clones":0,"num_object_copies":-8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":-297,"num_objects_unfound":0,"num_objects_dirty":-4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":1216512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008418"},"pg_stats":[{"pgid":"2.f","version":"48'10532","reported_seq":10847,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.293649+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:43.293649+0000","last_peered":"2024-09-15T21:22:43.293649+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:43.293649+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10532,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4730},{"shards":"0,2","objects":2091}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10641","reported_seq":10740,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.407804+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:44.407804+0000","last_peered":"2024-09-15T21:22:44.407804+0000","last_clean":"2024-09-15T21:22:44.407804+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:44.407804+0000","last_undegraded":"2024-09-15T21:22:44.407804+0000","last_fullsized":"2024-09-15T21:22:44.407804+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9367,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10710","reported_seq":17630,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.139111+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:43.139111+0000","last_peered":"2024-09-15T21:22:43.139111+0000","last_clean":"2024-09-15T21:22:43.139111+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:43.139111+0000","last_undegraded":"2024-09-15T21:22:43.139111+0000","last_fullsized":"2024-09-15T21:22:43.139111+0000","mapping_epoch":45,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419288,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9433,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10609","reported_seq":10683,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:44.379590+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:44.379590+0000","last_peered":"2024-09-15T21:22:44.379590+0000","last_clean":"2024-09-15T21:22:44.379590+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:44.379590+0000","last_undegraded":"2024-09-15T21:22:44.379590+0000","last_fullsized":"2024-09-15T21:22:44.379590+0000","mapping_epoch":36,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32608256,"num_objects":7961,"num_object_clones":0,"num_object_copies":15922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10652","reported_seq":10775,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.243479+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:43.243479+0000","last_peered":"2024-09-15T21:22:43.243479+0000","last_clean":"2024-09-15T21:22:43.243479+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:43.243479+0000","last_undegraded":"2024-09-15T21:22:43.243479+0000","last_fullsized":"2024-09-15T21:22:43.243479+0000","mapping_epoch":35,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33058816,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10462","reported_seq":10802,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.183886+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:43.183886+0000","last_peered":"2024-09-15T21:22:43.183886+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:43.183886+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436224,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7887,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4651},{"shards":"0,2","objects":2102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10716","reported_seq":11024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317908+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:43.317908+0000","last_peered":"2024-09-15T21:22:43.317908+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:43.317908+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538048,"num_objects":8188,"num_object_clones":0,"num_object_copies":16376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8352,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4807},{"shards":"0,2","objects":2208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10499","reported_seq":10623,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.249521+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:43.249521+0000","last_peered":"2024-09-15T21:22:43.249521+0000","last_clean":"2024-09-15T21:22:43.249521+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:43.249521+0000","last_undegraded":"2024-09-15T21:22:43.249521+0000","last_fullsized":"2024-09-15T21:22:43.249521+0000","mapping_epoch":35,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8284,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10470","reported_seq":13215,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415839+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:44.415839+0000","last_peered":"2024-09-15T21:22:44.415839+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:44.415839+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1308,"num_bytes_recovered":5324800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1909},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10275","reported_seq":10587,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.317294+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:43.317294+0000","last_peered":"2024-09-15T21:22:43.317294+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:43.317294+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'200","ondisk_log_start":"37'200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7801,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4512},{"shards":"0,2","objects":2087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10430","reported_seq":10670,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.360454+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:44.360454+0000","last_peered":"2024-09-15T21:22:44.360454+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:44.360454+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32174080,"num_objects":7855,"num_object_clones":0,"num_object_copies":15710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8981,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1981},{"shards":"2","objects":4652}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551213+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551213+0000","last_peered":"2024-09-15T21:22:00.551213+0000","last_clean":"2024-09-15T21:22:00.551213+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551213+0000","last_undegraded":"2024-09-15T21:22:00.551213+0000","last_fullsized":"2024-09-15T21:22:00.551213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10589","reported_seq":10816,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:44.415886+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:44.415886+0000","last_peered":"2024-09-15T21:22:44.415886+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:44.415886+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32600064,"num_objects":7959,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2038},{"shards":"2","objects":4663}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10625","reported_seq":10967,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.253926+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:43.253926+0000","last_peered":"2024-09-15T21:22:43.253926+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:43.253926+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33116160,"num_objects":8085,"num_object_clones":0,"num_object_copies":16170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7932,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4757},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551913+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551913+0000","last_peered":"2024-09-15T21:22:00.551913+0000","last_clean":"2024-09-15T21:22:00.551913+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551913+0000","last_undegraded":"2024-09-15T21:22:00.551913+0000","last_fullsized":"2024-09-15T21:22:00.551913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10506","reported_seq":10806,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.222903+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:43.222903+0000","last_peered":"2024-09-15T21:22:43.222903+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:43.222903+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32382976,"num_objects":7906,"num_object_clones":0,"num_object_copies":15812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8091,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4610},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10430","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.290945+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:43.290945+0000","last_peered":"2024-09-15T21:22:43.290945+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:43.290945+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8077,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4676},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10634","reported_seq":17723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.207392+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:43.207392+0000","last_peered":"2024-09-15T21:22:43.207392+0000","last_clean":"2024-09-15T21:22:43.207392+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:43.207392+0000","last_undegraded":"2024-09-15T21:22:43.207392+0000","last_fullsized":"2024-09-15T21:22:43.207392+0000","mapping_epoch":47,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32703081,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8294,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":522580609,"num_objects":127585,"num_object_clones":0,"num_object_copies":255170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29680,"num_objects_misplaced":1909,"num_objects_unfound":0,"num_objects_dirty":127585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140194,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8976,"num_bytes_recovered":33431552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1126080512,"data_stored":1126006267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608980,"kb_used_data":364868,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93762860,"statfs":{"total":96636764160,"available":96013168640,"internally_reserved":0,"allocated":373624832,"data_stored":373344470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149765,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788352,"kb_used_data":437420,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93583488,"statfs":{"total":96636764160,"available":95829491712,"internally_reserved":0,"allocated":447918080,"data_stored":447641790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,16,0,0,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626483,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566316,"kb_used_data":299996,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93805524,"statfs":{"total":96636764160,"available":96056856576,"internally_reserved":0,"allocated":307195904,"data_stored":306859878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446955520,"data_stored":446952041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306733056,"data_stored":306669841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372391936,"data_stored":372384385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:50.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:51.154 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:51.155 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:51.191 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-15T21:22:50.424408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514757268,"num_objects":125677,"num_object_clones":0,"num_object_copies":251354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142106,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9263,"num_bytes_recovered":34275347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948760,"kb_used_data":1091552,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":281166760,"statfs":{"total":289910292480,"available":287914762240,"internally_reserved":0,"allocated":1117749248,"data_stored":1116856570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":877710346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,15,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-182,"num_objects_misplaced":-503,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2060288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008393"},"pg_stats":[{"pgid":"2.f","version":"48'10653","reported_seq":10982,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.296726+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:48.296726+0000","last_peered":"2024-09-15T21:22:48.296726+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:48.296726+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190464,"num_objects":7859,"num_object_clones":0,"num_object_copies":15718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10653,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4616},{"shards":"0,2","objects":1970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10746","reported_seq":10847,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.024265+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:49.024265+0000","last_peered":"2024-09-15T21:22:49.024265+0000","last_clean":"2024-09-15T21:22:49.024265+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:49.024265+0000","last_undegraded":"2024-09-15T21:22:49.024265+0000","last_fullsized":"2024-09-15T21:22:49.024265+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9472,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10825","reported_seq":17745,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.104988+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:48.104988+0000","last_peered":"2024-09-15T21:22:48.104988+0000","last_clean":"2024-09-15T21:22:48.104988+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:48.104988+0000","last_undegraded":"2024-09-15T21:22:48.104988+0000","last_fullsized":"2024-09-15T21:22:48.104988+0000","mapping_epoch":45,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948248,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9548,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10716","reported_seq":10790,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.427109+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:49.427109+0000","last_peered":"2024-09-15T21:22:49.427109+0000","last_clean":"2024-09-15T21:22:49.427109+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:49.427109+0000","last_undegraded":"2024-09-15T21:22:49.427109+0000","last_fullsized":"2024-09-15T21:22:49.427109+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10716,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10776","reported_seq":10899,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.056042+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:48.056042+0000","last_peered":"2024-09-15T21:22:48.056042+0000","last_clean":"2024-09-15T21:22:48.056042+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:48.056042+0000","last_undegraded":"2024-09-15T21:22:48.056042+0000","last_fullsized":"2024-09-15T21:22:48.056042+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8634,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10599","reported_seq":10991,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313419+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:48.313419+0000","last_peered":"2024-09-15T21:22:48.313419+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:48.313419+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8024,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10840","reported_seq":11158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.111573+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:48.111573+0000","last_peered":"2024-09-15T21:22:48.111573+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:48.111573+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33030144,"num_objects":8064,"num_object_clones":0,"num_object_copies":16128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4688},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10599","reported_seq":10723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.036591+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:48.036591+0000","last_peered":"2024-09-15T21:22:48.036591+0000","last_clean":"2024-09-15T21:22:48.036591+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:48.036591+0000","last_undegraded":"2024-09-15T21:22:48.036591+0000","last_fullsized":"2024-09-15T21:22:48.036591+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969280,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10583","reported_seq":13741,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.401491+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:49.401491+0000","last_peered":"2024-09-15T21:22:49.401491+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:49.401491+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1514,"num_bytes_recovered":6168576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1703},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10401","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.083079+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:48.083079+0000","last_peered":"2024-09-15T21:22:48.083079+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:48.083079+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4400},{"shards":"0,2","objects":1961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10539","reported_seq":10781,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.360072+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:49.360072+0000","last_peered":"2024-09-15T21:22:49.360072+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:49.360072+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9090,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1872},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10697","reported_seq":10930,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.404434+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:49.404434+0000","last_peered":"2024-09-15T21:22:49.404434+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:49.404434+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":4558}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10737","reported_seq":11088,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313519+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:48.313519+0000","last_peered":"2024-09-15T21:22:48.313519+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:48.313519+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8044,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":2052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10634","reported_seq":10952,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.088262+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:48.088262+0000","last_peered":"2024-09-15T21:22:48.088262+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:48.088262+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8219,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4491},{"shards":"0,2","objects":1955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10571","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.280470+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:48.280470+0000","last_peered":"2024-09-15T21:22:48.280470+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:48.280470+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4546},{"shards":"0,2","objects":1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10774","reported_seq":17863,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.122110+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:48.122110+0000","last_peered":"2024-09-15T21:22:48.122110+0000","last_clean":"2024-09-15T21:22:48.122110+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:48.122110+0000","last_undegraded":"2024-09-15T21:22:48.122110+0000","last_fullsized":"2024-09-15T21:22:48.122110+0000","mapping_epoch":47,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129641,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514757249,"num_objects":125675,"num_object_clones":0,"num_object_copies":251350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142104,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":34275328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1115590656,"data_stored":1115516411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609512,"kb_used_data":361240,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93762328,"statfs":{"total":96636764160,"available":96012623872,"internally_reserved":0,"allocated":369909760,"data_stored":369629398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769632,"kb_used_data":431176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93602208,"statfs":{"total":96636764160,"available":95848660992,"internally_reserved":0,"allocated":441524224,"data_stored":441247934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,13,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569616,"kb_used_data":299136,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93802224,"statfs":{"total":96636764160,"available":96053477376,"internally_reserved":0,"allocated":306315264,"data_stored":305979238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440823808,"data_stored":440820329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305917952,"data_stored":305854737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":368848896,"data_stored":368841345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:51.192 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:51.617 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:51.652 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-15T21:22:50.424408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514757268,"num_objects":125677,"num_object_clones":0,"num_object_copies":251354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142106,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9263,"num_bytes_recovered":34275347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948760,"kb_used_data":1091552,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":281166760,"statfs":{"total":289910292480,"available":287914762240,"internally_reserved":0,"allocated":1117749248,"data_stored":1116856570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":877710346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,15,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-182,"num_objects_misplaced":-503,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":2060288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008393"},"pg_stats":[{"pgid":"2.f","version":"48'10653","reported_seq":10982,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.296726+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:48.296726+0000","last_peered":"2024-09-15T21:22:48.296726+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:48.296726+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190464,"num_objects":7859,"num_object_clones":0,"num_object_copies":15718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10653,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4616},{"shards":"0,2","objects":1970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10746","reported_seq":10847,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.024265+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:49.024265+0000","last_peered":"2024-09-15T21:22:49.024265+0000","last_clean":"2024-09-15T21:22:49.024265+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:49.024265+0000","last_undegraded":"2024-09-15T21:22:49.024265+0000","last_fullsized":"2024-09-15T21:22:49.024265+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9472,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10825","reported_seq":17745,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.104988+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:48.104988+0000","last_peered":"2024-09-15T21:22:48.104988+0000","last_clean":"2024-09-15T21:22:48.104988+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:48.104988+0000","last_undegraded":"2024-09-15T21:22:48.104988+0000","last_fullsized":"2024-09-15T21:22:48.104988+0000","mapping_epoch":45,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948248,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9548,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10716","reported_seq":10790,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.427109+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:49.427109+0000","last_peered":"2024-09-15T21:22:49.427109+0000","last_clean":"2024-09-15T21:22:49.427109+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:49.427109+0000","last_undegraded":"2024-09-15T21:22:49.427109+0000","last_fullsized":"2024-09-15T21:22:49.427109+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10716,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10776","reported_seq":10899,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.056042+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:48.056042+0000","last_peered":"2024-09-15T21:22:48.056042+0000","last_clean":"2024-09-15T21:22:48.056042+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:48.056042+0000","last_undegraded":"2024-09-15T21:22:48.056042+0000","last_fullsized":"2024-09-15T21:22:48.056042+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8634,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10599","reported_seq":10991,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313419+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:48.313419+0000","last_peered":"2024-09-15T21:22:48.313419+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:48.313419+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8024,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10840","reported_seq":11158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.111573+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:48.111573+0000","last_peered":"2024-09-15T21:22:48.111573+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:48.111573+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33030144,"num_objects":8064,"num_object_clones":0,"num_object_copies":16128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4688},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10599","reported_seq":10723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.036591+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:48.036591+0000","last_peered":"2024-09-15T21:22:48.036591+0000","last_clean":"2024-09-15T21:22:48.036591+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:48.036591+0000","last_undegraded":"2024-09-15T21:22:48.036591+0000","last_fullsized":"2024-09-15T21:22:48.036591+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969280,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10583","reported_seq":13741,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.401491+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:49.401491+0000","last_peered":"2024-09-15T21:22:49.401491+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:49.401491+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1514,"num_bytes_recovered":6168576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1703},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10401","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.083079+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:48.083079+0000","last_peered":"2024-09-15T21:22:48.083079+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:48.083079+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4400},{"shards":"0,2","objects":1961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10539","reported_seq":10781,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.360072+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:49.360072+0000","last_peered":"2024-09-15T21:22:49.360072+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:49.360072+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9090,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1872},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10697","reported_seq":10930,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.404434+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:49.404434+0000","last_peered":"2024-09-15T21:22:49.404434+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:49.404434+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":4558}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10737","reported_seq":11088,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313519+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:48.313519+0000","last_peered":"2024-09-15T21:22:48.313519+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:48.313519+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8044,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":2052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10634","reported_seq":10952,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.088262+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:48.088262+0000","last_peered":"2024-09-15T21:22:48.088262+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:48.088262+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8219,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4491},{"shards":"0,2","objects":1955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10571","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.280470+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:48.280470+0000","last_peered":"2024-09-15T21:22:48.280470+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:48.280470+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4546},{"shards":"0,2","objects":1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10774","reported_seq":17863,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.122110+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:48.122110+0000","last_peered":"2024-09-15T21:22:48.122110+0000","last_clean":"2024-09-15T21:22:48.122110+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:48.122110+0000","last_undegraded":"2024-09-15T21:22:48.122110+0000","last_fullsized":"2024-09-15T21:22:48.122110+0000","mapping_epoch":47,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129641,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514757249,"num_objects":125675,"num_object_clones":0,"num_object_copies":251350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142104,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":34275328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1115590656,"data_stored":1115516411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609512,"kb_used_data":361240,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93762328,"statfs":{"total":96636764160,"available":96012623872,"internally_reserved":0,"allocated":369909760,"data_stored":369629398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769632,"kb_used_data":431176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93602208,"statfs":{"total":96636764160,"available":95848660992,"internally_reserved":0,"allocated":441524224,"data_stored":441247934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,13,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569616,"kb_used_data":299136,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93802224,"statfs":{"total":96636764160,"available":96053477376,"internally_reserved":0,"allocated":306315264,"data_stored":305979238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440823808,"data_stored":440820329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305917952,"data_stored":305854737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":368848896,"data_stored":368841345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10653", 'reported_seq': 10982, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.296726+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:48.296726+0000', 'last_peered': '2024-09-15T21:22:48.296726+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:48.296726+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32190464, 'num_objects': 7859, 'num_object_clones': 0, 'num_object_copies': 15718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3243, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10653, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 124, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4616}, {'shards': '0,2', 'objects': 1970}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10599", 'reported_seq': 10991, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.313419+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:48.313419+0000', 'last_peered': '2024-09-15T21:22:48.313419+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:48.313419+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31875072, 'num_objects': 7782, 'num_object_clones': 0, 'num_object_copies': 15564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3244, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8024, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 129, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4540}, {'shards': '0,2', 'objects': 1965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10840", 'reported_seq': 11158, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.111573+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:48.111573+0000', 'last_peered': '2024-09-15T21:22:48.111573+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:48.111573+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33030144, 'num_objects': 8064, 'num_object_clones': 0, 'num_object_copies': 16128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3378, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8476, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 95, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4688}, {'shards': '0,2', 'objects': 2084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:51.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10583", 'reported_seq': 13741, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:49.401491+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:49.401491+0000', 'last_peered': '2024-09-15T21:22:49.401491+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:49.401491+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31916032, 'num_objects': 7792, 'num_object_clones': 0, 'num_object_copies': 15584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 1703, 'num_objects_unfound': 0, 'num_objects_dirty': 7792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9169, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1514, 'num_bytes_recovered': 6168576, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1703}, {'shards': '2', 'objects': 5966}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10401", 'reported_seq': 10744, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.083079+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:48.083079+0000', 'last_peered': '2024-09-15T21:22:48.083079+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:48.083079+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'400", 'ondisk_log_start': "37'400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31264768, 'num_objects': 7633, 'num_object_clones': 0, 'num_object_copies': 15266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3235, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7927, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 108, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4400}, {'shards': '0,2', 'objects': 1961}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10539", 'reported_seq': 10781, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:49.360072+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:49.360072+0000', 'last_peered': '2024-09-15T21:22:49.360072+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:49.360072+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31727616, 'num_objects': 7746, 'num_object_clones': 0, 'num_object_copies': 15492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3204, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9090, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1872}, {'shards': '2', 'objects': 4544}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10697", 'reported_seq': 10930, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:49.404434+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:49.404434+0000', 'last_peered': '2024-09-15T21:22:49.404434+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:49.404434+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32157696, 'num_objects': 7851, 'num_object_clones': 0, 'num_object_copies': 15702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3295, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9096, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1930}, {'shards': '2', 'objects': 4558}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10737", 'reported_seq': 11088, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.313519+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:48.313519+0000', 'last_peered': '2024-09-15T21:22:48.313519+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:48.313519+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32657408, 'num_objects': 7973, 'num_object_clones': 0, 'num_object_copies': 15946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8044, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 109, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4649}, {'shards': '0,2', 'objects': 2052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:51.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10634", 'reported_seq': 10952, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.088262+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:48.088262+0000', 'last_peered': '2024-09-15T21:22:48.088262+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:48.088262+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31858688, 'num_objects': 7778, 'num_object_clones': 0, 'num_object_copies': 15556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3290, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8219, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 92, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4491}, {'shards': '0,2', 'objects': 1955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:51.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10571", 'reported_seq': 10907, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.280470+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:48.280470+0000', 'last_peered': '2024-09-15T21:22:48.280470+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:48.280470+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31977472, 'num_objects': 7807, 'num_object_clones': 0, 'num_object_copies': 15614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8218, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 104, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4546}, {'shards': '0,2', 'objects': 1985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:52.435 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:52.435 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29600/251354 objects degraded (11.776%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":385,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125677,"data_bytes":514757268,"bytes_used":1995530240,"bytes_avail":287914762240,"bytes_total":289910292480,"degraded_objects":29600,"degraded_total":251354,"degraded_ratio":0.11776219992520509,"misplaced_objects":1703,"misplaced_total":251354,"misplaced_ratio":0.0067753049483994686,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":171570,"recovering_keys_per_sec":0,"num_objects_recovered":687,"num_bytes_recovered":2060288,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:52.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:52.487 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:53.242 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:53.247 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:53.285 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-15T21:22:52.424863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":514757268,"num_objects":125677,"num_object_clones":0,"num_object_copies":251354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125677,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142106,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9263,"num_bytes_recovered":34275347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948760,"kb_used_data":1091552,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":281166760,"statfs":{"total":289910292480,"available":287914762240,"internally_reserved":0,"allocated":1117749248,"data_stored":1116856570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":877710346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,15,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16711680,"num_objects":-4080,"num_object_clones":0,"num_object_copies":-8160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":-4080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002748"},"pg_stats":[{"pgid":"2.f","version":"48'10653","reported_seq":10982,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.296726+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:48.296726+0000","last_peered":"2024-09-15T21:22:48.296726+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:48.296726+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190464,"num_objects":7859,"num_object_clones":0,"num_object_copies":15718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10653,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4616},{"shards":"0,2","objects":1970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10746","reported_seq":10847,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.024265+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:49.024265+0000","last_peered":"2024-09-15T21:22:49.024265+0000","last_clean":"2024-09-15T21:22:49.024265+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:49.024265+0000","last_undegraded":"2024-09-15T21:22:49.024265+0000","last_fullsized":"2024-09-15T21:22:49.024265+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9472,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10825","reported_seq":17745,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.104988+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:48.104988+0000","last_peered":"2024-09-15T21:22:48.104988+0000","last_clean":"2024-09-15T21:22:48.104988+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:48.104988+0000","last_undegraded":"2024-09-15T21:22:48.104988+0000","last_fullsized":"2024-09-15T21:22:48.104988+0000","mapping_epoch":45,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32948248,"num_objects":8045,"num_object_clones":0,"num_object_copies":16090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9548,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10716","reported_seq":10790,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:49.427109+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:49.427109+0000","last_peered":"2024-09-15T21:22:49.427109+0000","last_clean":"2024-09-15T21:22:49.427109+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:49.427109+0000","last_undegraded":"2024-09-15T21:22:49.427109+0000","last_fullsized":"2024-09-15T21:22:49.427109+0000","mapping_epoch":36,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10716,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10776","reported_seq":10899,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.056042+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:48.056042+0000","last_peered":"2024-09-15T21:22:48.056042+0000","last_clean":"2024-09-15T21:22:48.056042+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:48.056042+0000","last_undegraded":"2024-09-15T21:22:48.056042+0000","last_fullsized":"2024-09-15T21:22:48.056042+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8634,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10599","reported_seq":10991,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313419+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:48.313419+0000","last_peered":"2024-09-15T21:22:48.313419+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:48.313419+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8024,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10840","reported_seq":11158,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.111573+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:48.111573+0000","last_peered":"2024-09-15T21:22:48.111573+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:48.111573+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33030144,"num_objects":8064,"num_object_clones":0,"num_object_copies":16128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8476,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4688},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10599","reported_seq":10723,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.036591+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:48.036591+0000","last_peered":"2024-09-15T21:22:48.036591+0000","last_clean":"2024-09-15T21:22:48.036591+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:48.036591+0000","last_undegraded":"2024-09-15T21:22:48.036591+0000","last_fullsized":"2024-09-15T21:22:48.036591+0000","mapping_epoch":35,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969280,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10583","reported_seq":13741,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.401491+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:49.401491+0000","last_peered":"2024-09-15T21:22:49.401491+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:49.401491+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31916032,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1514,"num_bytes_recovered":6168576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1703},{"shards":"2","objects":5966}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10401","reported_seq":10744,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.083079+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:48.083079+0000","last_peered":"2024-09-15T21:22:48.083079+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:48.083079+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'400","ondisk_log_start":"37'400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7927,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4400},{"shards":"0,2","objects":1961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10539","reported_seq":10781,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.360072+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:49.360072+0000","last_peered":"2024-09-15T21:22:49.360072+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:49.360072+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31727616,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9090,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1872},{"shards":"2","objects":4544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10697","reported_seq":10930,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:49.404434+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:49.404434+0000","last_peered":"2024-09-15T21:22:49.404434+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:49.404434+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":4558}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10737","reported_seq":11088,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.313519+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:48.313519+0000","last_peered":"2024-09-15T21:22:48.313519+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:48.313519+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8044,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":2052}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10634","reported_seq":10952,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.088262+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:48.088262+0000","last_peered":"2024-09-15T21:22:48.088262+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:48.088262+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31858688,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8219,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4491},{"shards":"0,2","objects":1955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10571","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.280470+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:48.280470+0000","last_peered":"2024-09-15T21:22:48.280470+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:48.280470+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8218,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4546},{"shards":"0,2","objects":1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10774","reported_seq":17863,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.122110+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:48.122110+0000","last_peered":"2024-09-15T21:22:48.122110+0000","last_clean":"2024-09-15T21:22:48.122110+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:48.122110+0000","last_undegraded":"2024-09-15T21:22:48.122110+0000","last_fullsized":"2024-09-15T21:22:48.122110+0000","mapping_epoch":47,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129641,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":514757249,"num_objects":125675,"num_object_clones":0,"num_object_copies":251350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29600,"num_objects_misplaced":1703,"num_objects_unfound":0,"num_objects_dirty":125675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142104,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9262,"num_bytes_recovered":34275328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1115590656,"data_stored":1115516411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609512,"kb_used_data":361240,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93762328,"statfs":{"total":96636764160,"available":96012623872,"internally_reserved":0,"allocated":369909760,"data_stored":369629398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149766,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769632,"kb_used_data":431176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93602208,"statfs":{"total":96636764160,"available":95848660992,"internally_reserved":0,"allocated":441524224,"data_stored":441247934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,13,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569616,"kb_used_data":299136,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93802224,"statfs":{"total":96636764160,"available":96053477376,"internally_reserved":0,"allocated":306315264,"data_stored":305979238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":276950700},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440823808,"data_stored":440820329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305917952,"data_stored":305854737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":368848896,"data_stored":368841345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:56.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:56.948 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:56.986 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T21:22:56.425808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506675860,"num_objects":123704,"num_object_clones":0,"num_object_copies":247408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29453,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":123704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144079,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9587,"num_bytes_recovered":35000339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1938644,"kb_used_data":1081432,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":281176876,"statfs":{"total":289910292480,"available":287925121024,"internally_reserved":0,"allocated":1107386368,"data_stored":1106493690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":877710346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,4,5,5,10,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15904768,"num_objects":-3883,"num_object_clones":0,"num_object_copies":-7766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-227,"num_objects_misplaced":-383,"num_objects_unfound":0,"num_objects_dirty":-3883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":1568768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002812"},"pg_stats":[{"pgid":"2.f","version":"48'10761","reported_seq":11118,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.076753+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:53.076753+0000","last_peered":"2024-09-15T21:22:53.076753+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:53.076753+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31748096,"num_objects":7751,"num_object_clones":0,"num_object_copies":15502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10761,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":1862}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10874","reported_seq":10975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.459315+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:54.459315+0000","last_peered":"2024-09-15T21:22:54.459315+0000","last_clean":"2024-09-15T21:22:54.459315+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:54.459315+0000","last_undegraded":"2024-09-15T21:22:54.459315+0000","last_fullsized":"2024-09-15T21:22:54.459315+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9600,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10936","reported_seq":17856,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.268449+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:53.268449+0000","last_peered":"2024-09-15T21:22:53.268449+0000","last_clean":"2024-09-15T21:22:53.268449+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:53.268449+0000","last_undegraded":"2024-09-15T21:22:53.268449+0000","last_fullsized":"2024-09-15T21:22:53.268449+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32493592,"num_objects":7934,"num_object_clones":0,"num_object_copies":15868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9659,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10859","reported_seq":10933,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.428844+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:54.428844+0000","last_peered":"2024-09-15T21:22:54.428844+0000","last_clean":"2024-09-15T21:22:54.428844+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:54.428844+0000","last_undegraded":"2024-09-15T21:22:54.428844+0000","last_fullsized":"2024-09-15T21:22:54.428844+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31584256,"num_objects":7711,"num_object_clones":0,"num_object_copies":15422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10859,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10899","reported_seq":11022,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.236595+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:53.236595+0000","last_peered":"2024-09-15T21:22:53.236595+0000","last_clean":"2024-09-15T21:22:53.236595+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:53.236595+0000","last_undegraded":"2024-09-15T21:22:53.236595+0000","last_fullsized":"2024-09-15T21:22:53.236595+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10736","reported_seq":11174,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.295947+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:53.295947+0000","last_peered":"2024-09-15T21:22:53.295947+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:53.295947+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31313920,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8161,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":1828}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10967","reported_seq":11335,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.324919+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:53.324919+0000","last_peered":"2024-09-15T21:22:53.324919+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:53.324919+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32509952,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8603,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4586},{"shards":"0,2","objects":1957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10716","reported_seq":10840,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.300909+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:53.300909+0000","last_peered":"2024-09-15T21:22:53.300909+0000","last_clean":"2024-09-15T21:22:53.300909+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:53.300909+0000","last_undegraded":"2024-09-15T21:22:53.300909+0000","last_fullsized":"2024-09-15T21:22:53.300909+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8501,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10710","reported_seq":14222,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.437581+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:54.437581+0000","last_peered":"2024-09-15T21:22:54.437581+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:54.437581+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":6893568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1526},{"shards":"2","objects":6016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10522","reported_seq":10915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.311162+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:53.311162+0000","last_peered":"2024-09-15T21:22:53.311162+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:53.311162+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30769152,"num_objects":7512,"num_object_clones":0,"num_object_copies":15024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8048,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":1840}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10660","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.388350+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:54.388350+0000","last_peered":"2024-09-15T21:22:54.388350+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:54.388350+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31232000,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1751},{"shards":"2","objects":4427}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10829","reported_seq":11076,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.419848+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:54.419848+0000","last_peered":"2024-09-15T21:22:54.419848+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:54.419848+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31617024,"num_objects":7719,"num_object_clones":0,"num_object_copies":15438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9228,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1798},{"shards":"2","objects":4433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10862","reported_seq":11253,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.339271+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:53.339271+0000","last_peered":"2024-09-15T21:22:53.339271+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:53.339271+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32145408,"num_objects":7848,"num_object_clones":0,"num_object_copies":15696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8169,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":1927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10751","reported_seq":11097,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.344468+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:53.344468+0000","last_peered":"2024-09-15T21:22:53.344468+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:53.344468+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379456,"num_objects":7661,"num_object_clones":0,"num_object_copies":15322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4388},{"shards":"0,2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10689","reported_seq":11055,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.317800+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:53.317800+0000","last_peered":"2024-09-15T21:22:53.317800+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:53.317800+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4443},{"shards":"0,2","objects":1867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10892","reported_seq":17981,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.291778+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:53.291778+0000","last_peered":"2024-09-15T21:22:53.291778+0000","last_clean":"2024-09-15T21:22:53.291778+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:53.291778+0000","last_undegraded":"2024-09-15T21:22:53.291778+0000","last_fullsized":"2024-09-15T21:22:53.291778+0000","mapping_epoch":47,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31646313,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8552,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506675841,"num_objects":123702,"num_object_clones":0,"num_object_copies":247404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29453,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":123702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144077,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9586,"num_bytes_recovered":35000320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104732160,"data_stored":1104657915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609768,"kb_used_data":357336,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93762072,"statfs":{"total":96636764160,"available":96012361728,"internally_reserved":0,"allocated":365912064,"data_stored":365631702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,1,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768132,"kb_used_data":425512,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93603708,"statfs":{"total":96636764160,"available":95850196992,"internally_reserved":0,"allocated":435724288,"data_stored":435447998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,3,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560744,"kb_used_data":298584,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93811096,"statfs":{"total":96636764160,"available":96062562304,"internally_reserved":0,"allocated":305750016,"data_stored":305413990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434806784,"data_stored":434803305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305258496,"data_stored":305195281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364666880,"data_stored":364659329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:56.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:57.821 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T21:22:56.425808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506675860,"num_objects":123704,"num_object_clones":0,"num_object_copies":247408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29453,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":123704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144079,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9587,"num_bytes_recovered":35000339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1938644,"kb_used_data":1081432,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":281176876,"statfs":{"total":289910292480,"available":287925121024,"internally_reserved":0,"allocated":1107386368,"data_stored":1106493690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":877710346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,4,5,5,10,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15904768,"num_objects":-3883,"num_object_clones":0,"num_object_copies":-7766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-227,"num_objects_misplaced":-383,"num_objects_unfound":0,"num_objects_dirty":-3883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":1568768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002812"},"pg_stats":[{"pgid":"2.f","version":"48'10761","reported_seq":11118,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.076753+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:53.076753+0000","last_peered":"2024-09-15T21:22:53.076753+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:53.076753+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31748096,"num_objects":7751,"num_object_clones":0,"num_object_copies":15502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10761,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":1862}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10874","reported_seq":10975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.459315+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:54.459315+0000","last_peered":"2024-09-15T21:22:54.459315+0000","last_clean":"2024-09-15T21:22:54.459315+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:54.459315+0000","last_undegraded":"2024-09-15T21:22:54.459315+0000","last_fullsized":"2024-09-15T21:22:54.459315+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9600,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10936","reported_seq":17856,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.268449+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:53.268449+0000","last_peered":"2024-09-15T21:22:53.268449+0000","last_clean":"2024-09-15T21:22:53.268449+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:53.268449+0000","last_undegraded":"2024-09-15T21:22:53.268449+0000","last_fullsized":"2024-09-15T21:22:53.268449+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32493592,"num_objects":7934,"num_object_clones":0,"num_object_copies":15868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9659,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10859","reported_seq":10933,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.428844+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:54.428844+0000","last_peered":"2024-09-15T21:22:54.428844+0000","last_clean":"2024-09-15T21:22:54.428844+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:54.428844+0000","last_undegraded":"2024-09-15T21:22:54.428844+0000","last_fullsized":"2024-09-15T21:22:54.428844+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31584256,"num_objects":7711,"num_object_clones":0,"num_object_copies":15422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10859,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'10899","reported_seq":11022,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.236595+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:53.236595+0000","last_peered":"2024-09-15T21:22:53.236595+0000","last_clean":"2024-09-15T21:22:53.236595+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:53.236595+0000","last_undegraded":"2024-09-15T21:22:53.236595+0000","last_fullsized":"2024-09-15T21:22:53.236595+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10736","reported_seq":11174,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.295947+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:53.295947+0000","last_peered":"2024-09-15T21:22:53.295947+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:53.295947+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31313920,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8161,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4426},{"shards":"0,2","objects":1828}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'10967","reported_seq":11335,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.324919+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:53.324919+0000","last_peered":"2024-09-15T21:22:53.324919+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:53.324919+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32509952,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8603,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4586},{"shards":"0,2","objects":1957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10716","reported_seq":10840,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.300909+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:53.300909+0000","last_peered":"2024-09-15T21:22:53.300909+0000","last_clean":"2024-09-15T21:22:53.300909+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:53.300909+0000","last_undegraded":"2024-09-15T21:22:53.300909+0000","last_fullsized":"2024-09-15T21:22:53.300909+0000","mapping_epoch":35,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8501,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10710","reported_seq":14222,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.437581+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:54.437581+0000","last_peered":"2024-09-15T21:22:54.437581+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:54.437581+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":6893568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1526},{"shards":"2","objects":6016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10522","reported_seq":10915,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.311162+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:53.311162+0000","last_peered":"2024-09-15T21:22:53.311162+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:53.311162+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'500","ondisk_log_start":"37'500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30769152,"num_objects":7512,"num_object_clones":0,"num_object_copies":15024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8048,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4304},{"shards":"0,2","objects":1840}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10660","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.388350+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:54.388350+0000","last_peered":"2024-09-15T21:22:54.388350+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:54.388350+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31232000,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1751},{"shards":"2","objects":4427}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10829","reported_seq":11076,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.419848+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:54.419848+0000","last_peered":"2024-09-15T21:22:54.419848+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:54.419848+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31617024,"num_objects":7719,"num_object_clones":0,"num_object_copies":15438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9228,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1798},{"shards":"2","objects":4433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'10862","reported_seq":11253,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.339271+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:53.339271+0000","last_peered":"2024-09-15T21:22:53.339271+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:53.339271+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32145408,"num_objects":7848,"num_object_clones":0,"num_object_copies":15696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8169,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":1927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10751","reported_seq":11097,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.344468+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:53.344468+0000","last_peered":"2024-09-15T21:22:53.344468+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:53.344468+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379456,"num_objects":7661,"num_object_clones":0,"num_object_copies":15322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4388},{"shards":"0,2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10689","reported_seq":11055,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.317800+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:53.317800+0000","last_peered":"2024-09-15T21:22:53.317800+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:53.317800+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4443},{"shards":"0,2","objects":1867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10892","reported_seq":17981,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.291778+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:53.291778+0000","last_peered":"2024-09-15T21:22:53.291778+0000","last_clean":"2024-09-15T21:22:53.291778+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:53.291778+0000","last_undegraded":"2024-09-15T21:22:53.291778+0000","last_fullsized":"2024-09-15T21:22:53.291778+0000","mapping_epoch":47,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31646313,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8552,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506675841,"num_objects":123702,"num_object_clones":0,"num_object_copies":247404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29453,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":123702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144077,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9586,"num_bytes_recovered":35000320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1104732160,"data_stored":1104657915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609768,"kb_used_data":357336,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93762072,"statfs":{"total":96636764160,"available":96012361728,"internally_reserved":0,"allocated":365912064,"data_stored":365631702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,1,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149767,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768132,"kb_used_data":425512,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93603708,"statfs":{"total":96636764160,"available":95850196992,"internally_reserved":0,"allocated":435724288,"data_stored":435447998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,3,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560744,"kb_used_data":298584,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93811096,"statfs":{"total":96636764160,"available":96062562304,"internally_reserved":0,"allocated":305750016,"data_stored":305413990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434806784,"data_stored":434803305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305258496,"data_stored":305195281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364666880,"data_stored":364659329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:57.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:57.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10761", 'reported_seq': 11118, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.076753+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:53.076753+0000', 'last_peered': '2024-09-15T21:22:53.076753+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:53.076753+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31748096, 'num_objects': 7751, 'num_object_clones': 0, 'num_object_copies': 15502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3229, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10761, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 138, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4522}, {'shards': '0,2', 'objects': 1862}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10736", 'reported_seq': 11174, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.295947+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:53.295947+0000', 'last_peered': '2024-09-15T21:22:53.295947+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:53.295947+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31313920, 'num_objects': 7645, 'num_object_clones': 0, 'num_object_copies': 15290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3221, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8161, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4426}, {'shards': '0,2', 'objects': 1828}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'10967", 'reported_seq': 11335, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.324919+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:53.324919+0000', 'last_peered': '2024-09-15T21:22:53.324919+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:53.324919+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32509952, 'num_objects': 7937, 'num_object_clones': 0, 'num_object_copies': 15874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3353, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7937, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8603, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 120, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4586}, {'shards': '0,2', 'objects': 1957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10710", 'reported_seq': 14222, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:54.437581+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:54.437581+0000', 'last_peered': '2024-09-15T21:22:54.437581+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:54.437581+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31395840, 'num_objects': 7665, 'num_object_clones': 0, 'num_object_copies': 15330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 1526, 'num_objects_unfound': 0, 'num_objects_dirty': 7665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9296, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1691, 'num_bytes_recovered': 6893568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1526}, {'shards': '2', 'objects': 6016}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10522", 'reported_seq': 10915, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.311162+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:53.311162+0000', 'last_peered': '2024-09-15T21:22:53.311162+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:53.311162+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'500", 'ondisk_log_start': "37'500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30769152, 'num_objects': 7512, 'num_object_clones': 0, 'num_object_copies': 15024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3210, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8048, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 133, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4304}, {'shards': '0,2', 'objects': 1840}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:57.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10660", 'reported_seq': 10907, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:54.388350+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:54.388350+0000', 'last_peered': '2024-09-15T21:22:54.388350+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:54.388350+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31232000, 'num_objects': 7625, 'num_object_clones': 0, 'num_object_copies': 15250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3200, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7625, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9211, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1751}, {'shards': '2', 'objects': 4427}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10829", 'reported_seq': 11076, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:54.419848+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:54.419848+0000', 'last_peered': '2024-09-15T21:22:54.419848+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:54.419848+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31617024, 'num_objects': 7719, 'num_object_clones': 0, 'num_object_copies': 15438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3288, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9228, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1798}, {'shards': '2', 'objects': 4433}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'10862", 'reported_seq': 11253, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.339271+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:53.339271+0000', 'last_peered': '2024-09-15T21:22:53.339271+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:53.339271+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32145408, 'num_objects': 7848, 'num_object_clones': 0, 'num_object_copies': 15696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3305, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8169, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 129, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4544}, {'shards': '0,2', 'objects': 1927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10751", 'reported_seq': 11097, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.344468+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:53.344468+0000', 'last_peered': '2024-09-15T21:22:53.344468+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:53.344468+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31379456, 'num_objects': 7661, 'num_object_clones': 0, 'num_object_copies': 15322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3276, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8336, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 106, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4388}, {'shards': '0,2', 'objects': 1838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:57.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10689", 'reported_seq': 11055, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.317800+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:53.317800+0000', 'last_peered': '2024-09-15T21:22:53.317800+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:53.317800+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31494144, 'num_objects': 7689, 'num_object_clones': 0, 'num_object_copies': 15378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8336, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4443}, {'shards': '0,2', 'objects': 1867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:57.825 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:58.844 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29377/244904 objects degraded (11.995%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122452,"data_bytes":501547668,"bytes_used":1986879488,"bytes_avail":287923412992,"bytes_total":289910292480,"degraded_objects":29377,"degraded_total":244904,"degraded_ratio":0.11995312448959593,"misplaced_objects":1526,"misplaced_total":244904,"misplaced_ratio":0.0062310129683467805,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":130700,"recovering_keys_per_sec":0,"num_objects_recovered":686,"num_bytes_recovered":1568768,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":427},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:58.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:58.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:59.359 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:22:59.360 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:22:59.417 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-15T21:22:58.426272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":501547668,"num_objects":122452,"num_object_clones":0,"num_object_copies":244904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29377,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":122452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145331,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9663,"num_bytes_recovered":35000339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1940312,"kb_used_data":1074792,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":281175208,"statfs":{"total":289910292480,"available":287923412992,"internally_reserved":0,"allocated":1100587008,"data_stored":1099694330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":886230026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,6,5,6,0,0,0,10],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21032960,"num_objects":-5135,"num_object_clones":0,"num_object_copies":-10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":-383,"num_objects_unfound":0,"num_objects_dirty":-5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5135,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":1568768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002779"},"pg_stats":[{"pgid":"2.f","version":"48'10904","reported_seq":11281,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363632+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:58.363632+0000","last_peered":"2024-09-15T21:22:58.363632+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:58.363632+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'10874","reported_seq":10975,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.459315+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:54.459315+0000","last_peered":"2024-09-15T21:22:54.459315+0000","last_clean":"2024-09-15T21:22:54.459315+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:54.459315+0000","last_undegraded":"2024-09-15T21:22:54.459315+0000","last_fullsized":"2024-09-15T21:22:54.459315+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9600,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'10936","reported_seq":17856,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.268449+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:53.268449+0000","last_peered":"2024-09-15T21:22:53.268449+0000","last_clean":"2024-09-15T21:22:53.268449+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:53.268449+0000","last_undegraded":"2024-09-15T21:22:53.268449+0000","last_fullsized":"2024-09-15T21:22:53.268449+0000","mapping_epoch":45,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32493592,"num_objects":7934,"num_object_clones":0,"num_object_copies":15868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9659,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'10859","reported_seq":10933,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:54.428844+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:54.428844+0000","last_peered":"2024-09-15T21:22:54.428844+0000","last_clean":"2024-09-15T21:22:54.428844+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:54.428844+0000","last_undegraded":"2024-09-15T21:22:54.428844+0000","last_fullsized":"2024-09-15T21:22:54.428844+0000","mapping_epoch":36,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31584256,"num_objects":7711,"num_object_clones":0,"num_object_copies":15422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10859,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11050","reported_seq":11173,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.220092+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:58.220092+0000","last_peered":"2024-09-15T21:22:58.220092+0000","last_clean":"2024-09-15T21:22:58.220092+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:58.220092+0000","last_undegraded":"2024-09-15T21:22:58.220092+0000","last_fullsized":"2024-09-15T21:22:58.220092+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8908,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10870","reported_seq":11328,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.378770+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:58.378770+0000","last_peered":"2024-09-15T21:22:58.378770+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:58.378770+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4302},{"shards":"0,2","objects":1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11111","reported_seq":11506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.339576+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:58.339576+0000","last_peered":"2024-09-15T21:22:58.339576+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:58.339576+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31920128,"num_objects":7793,"num_object_clones":0,"num_object_copies":15586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8747,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4456},{"shards":"0,2","objects":1813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10872","reported_seq":10996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.294395+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:58.294395+0000","last_peered":"2024-09-15T21:22:58.294395+0000","last_clean":"2024-09-15T21:22:58.294395+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:58.294395+0000","last_undegraded":"2024-09-15T21:22:58.294395+0000","last_fullsized":"2024-09-15T21:22:58.294395+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10710","reported_seq":14222,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.437581+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:54.437581+0000","last_peered":"2024-09-15T21:22:54.437581+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:54.437581+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":6893568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1526},{"shards":"2","objects":6016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10651","reported_seq":11066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.332928+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:58.332928+0000","last_peered":"2024-09-15T21:22:58.332928+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:58.332928+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30240768,"num_objects":7383,"num_object_clones":0,"num_object_copies":14766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8177,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4186},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10660","reported_seq":10907,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.388350+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:54.388350+0000","last_peered":"2024-09-15T21:22:54.388350+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:54.388350+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31232000,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9211,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1751},{"shards":"2","objects":4427}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10829","reported_seq":11076,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:54.419848+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:54.419848+0000","last_peered":"2024-09-15T21:22:54.419848+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:54.419848+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31617024,"num_objects":7719,"num_object_clones":0,"num_object_copies":15438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9228,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1798},{"shards":"2","objects":4433}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11007","reported_seq":11416,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:58.363948+0000","last_peered":"2024-09-15T21:22:58.363948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:58.363948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8314,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":1782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10867","reported_seq":11235,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.255942+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:58.255942+0000","last_peered":"2024-09-15T21:22:58.255942+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:58.255942+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30904320,"num_objects":7545,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8452,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4283},{"shards":"0,2","objects":1722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10823","reported_seq":11211,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.317996+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:58.317996+0000","last_peered":"2024-09-15T21:22:58.317996+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:58.317996+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552581+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552581+0000","last_peered":"2024-09-15T21:22:00.552581+0000","last_clean":"2024-09-15T21:22:00.552581+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552581+0000","last_undegraded":"2024-09-15T21:22:00.552581+0000","last_fullsized":"2024-09-15T21:22:00.552581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'10892","reported_seq":17981,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.291778+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:53.291778+0000","last_peered":"2024-09-15T21:22:53.291778+0000","last_clean":"2024-09-15T21:22:53.291778+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:53.291778+0000","last_undegraded":"2024-09-15T21:22:53.291778+0000","last_fullsized":"2024-09-15T21:22:53.291778+0000","mapping_epoch":47,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31646313,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8552,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":501547649,"num_objects":122450,"num_object_clones":0,"num_object_copies":244900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29377,"num_objects_misplaced":1526,"num_objects_unfound":0,"num_objects_dirty":122450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145329,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9662,"num_bytes_recovered":35000320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1097842688,"data_stored":1097768443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953540,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609768,"kb_used_data":357336,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93762072,"statfs":{"total":96636764160,"available":96012361728,"internally_reserved":0,"allocated":365912064,"data_stored":365631702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,1,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769800,"kb_used_data":418872,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93602040,"statfs":{"total":96636764160,"available":95848488960,"internally_reserved":0,"allocated":428924928,"data_stored":428648638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,3,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560744,"kb_used_data":298584,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93811096,"statfs":{"total":96636764160,"available":96062562304,"internally_reserved":0,"allocated":305750016,"data_stored":305413990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":268431020},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427917312,"data_stored":427913833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305258496,"data_stored":305195281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364666880,"data_stored":364659329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:02.420 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:03.161 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:03.163 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:03.195 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-15T21:23:02.427229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":497635988,"num_objects":121497,"num_object_clones":0,"num_object_copies":242994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29368,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":121497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146286,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9839,"num_bytes_recovered":35684371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1943060,"kb_used_data":1069220,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":281172460,"statfs":{"total":289910292480,"available":287920599040,"internally_reserved":0,"allocated":1094881280,"data_stored":1093988602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":894749706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,7,4,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17121280,"num_objects":-4180,"num_object_clones":0,"num_object_copies":-8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":-4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4180,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"48'10904","reported_seq":11281,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363632+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:58.363632+0000","last_peered":"2024-09-15T21:22:58.363632+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:58.363632+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11003","reported_seq":11104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.410809+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:59.410809+0000","last_peered":"2024-09-15T21:22:59.410809+0000","last_clean":"2024-09-15T21:22:59.410809+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:59.410809+0000","last_undegraded":"2024-09-15T21:22:59.410809+0000","last_fullsized":"2024-09-15T21:22:59.410809+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31281152,"num_objects":7637,"num_object_clones":0,"num_object_copies":15274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11068","reported_seq":17988,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.394885+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:58.394885+0000","last_peered":"2024-09-15T21:22:58.394885+0000","last_clean":"2024-09-15T21:22:58.394885+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:58.394885+0000","last_undegraded":"2024-09-15T21:22:58.394885+0000","last_fullsized":"2024-09-15T21:22:58.394885+0000","mapping_epoch":45,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31952920,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11004","reported_seq":11078,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.401472+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:59.401472+0000","last_peered":"2024-09-15T21:22:59.401472+0000","last_clean":"2024-09-15T21:22:59.401472+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:59.401472+0000","last_undegraded":"2024-09-15T21:22:59.401472+0000","last_fullsized":"2024-09-15T21:22:59.401472+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11004,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11050","reported_seq":11173,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.220092+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:58.220092+0000","last_peered":"2024-09-15T21:22:58.220092+0000","last_clean":"2024-09-15T21:22:58.220092+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:58.220092+0000","last_undegraded":"2024-09-15T21:22:58.220092+0000","last_fullsized":"2024-09-15T21:22:58.220092+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8908,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10870","reported_seq":11328,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.378770+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:58.378770+0000","last_peered":"2024-09-15T21:22:58.378770+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:58.378770+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4302},{"shards":"0,2","objects":1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11111","reported_seq":11506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.339576+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:58.339576+0000","last_peered":"2024-09-15T21:22:58.339576+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:58.339576+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31920128,"num_objects":7793,"num_object_clones":0,"num_object_copies":15586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8747,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4456},{"shards":"0,2","objects":1813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10872","reported_seq":10996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.294395+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:58.294395+0000","last_peered":"2024-09-15T21:22:58.294395+0000","last_clean":"2024-09-15T21:22:58.294395+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:58.294395+0000","last_undegraded":"2024-09-15T21:22:58.294395+0000","last_fullsized":"2024-09-15T21:22:58.294395+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10841","reported_seq":14687,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.466403+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:59.466403+0000","last_peered":"2024-09-15T21:22:59.466403+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:59.466403+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9427,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1858,"num_bytes_recovered":7577600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1359},{"shards":"2","objects":6052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10651","reported_seq":11066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.332928+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:58.332928+0000","last_peered":"2024-09-15T21:22:58.332928+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:58.332928+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30240768,"num_objects":7383,"num_object_clones":0,"num_object_copies":14766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8177,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4186},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10784","reported_seq":11048,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.362309+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:59.362309+0000","last_peered":"2024-09-15T21:22:59.362309+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:59.362309+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1627},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10969","reported_seq":11220,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.373976+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:59.373976+0000","last_peered":"2024-09-15T21:22:59.373976+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:59.373976+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9368,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1658},{"shards":"2","objects":4295}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11007","reported_seq":11416,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:58.363948+0000","last_peered":"2024-09-15T21:22:58.363948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:58.363948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8314,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":1782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10867","reported_seq":11235,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.255942+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:58.255942+0000","last_peered":"2024-09-15T21:22:58.255942+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:58.255942+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30904320,"num_objects":7545,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8452,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4283},{"shards":"0,2","objects":1722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10823","reported_seq":11211,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.317996+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:58.317996+0000","last_peered":"2024-09-15T21:22:58.317996+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:58.317996+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11046","reported_seq":18135,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.374509+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:58.374509+0000","last_peered":"2024-09-15T21:22:58.374509+0000","last_clean":"2024-09-15T21:22:58.374509+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:58.374509+0000","last_undegraded":"2024-09-15T21:22:58.374509+0000","last_fullsized":"2024-09-15T21:22:58.374509+0000","mapping_epoch":47,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31015529,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":497635969,"num_objects":121495,"num_object_clones":0,"num_object_copies":242990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29368,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":121495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146284,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9838,"num_bytes_recovered":35684352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092243456,"data_stored":1092169211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609412,"kb_used_data":352820,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93762428,"statfs":{"total":96636764160,"available":96012726272,"internally_reserved":0,"allocated":361287680,"data_stored":361007318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769800,"kb_used_data":418872,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93602040,"statfs":{"total":96636764160,"available":95848488960,"internally_reserved":0,"allocated":428924928,"data_stored":428648638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,3,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563848,"kb_used_data":297528,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93807992,"statfs":{"total":96636764160,"available":96059383808,"internally_reserved":0,"allocated":304668672,"data_stored":304332646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427917312,"data_stored":427913833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304119808,"data_stored":304056593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360206336,"data_stored":360198785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:03.196 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:03.902 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:03.948 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-15T21:23:02.427229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":497635988,"num_objects":121497,"num_object_clones":0,"num_object_copies":242994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29368,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":121497,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146286,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9839,"num_bytes_recovered":35684371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1943060,"kb_used_data":1069220,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":281172460,"statfs":{"total":289910292480,"available":287920599040,"internally_reserved":0,"allocated":1094881280,"data_stored":1093988602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":894749706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,7,4,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17121280,"num_objects":-4180,"num_object_clones":0,"num_object_copies":-8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":-4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4180,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"48'10904","reported_seq":11281,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363632+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:22:58.363632+0000","last_peered":"2024-09-15T21:22:58.363632+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:22:58.363632+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":1719}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11003","reported_seq":11104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.410809+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:59.410809+0000","last_peered":"2024-09-15T21:22:59.410809+0000","last_clean":"2024-09-15T21:22:59.410809+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:59.410809+0000","last_undegraded":"2024-09-15T21:22:59.410809+0000","last_fullsized":"2024-09-15T21:22:59.410809+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31281152,"num_objects":7637,"num_object_clones":0,"num_object_copies":15274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11068","reported_seq":17988,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.394885+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:22:58.394885+0000","last_peered":"2024-09-15T21:22:58.394885+0000","last_clean":"2024-09-15T21:22:58.394885+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:22:58.394885+0000","last_undegraded":"2024-09-15T21:22:58.394885+0000","last_fullsized":"2024-09-15T21:22:58.394885+0000","mapping_epoch":45,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31952920,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9791,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11004","reported_seq":11078,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.401472+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:59.401472+0000","last_peered":"2024-09-15T21:22:59.401472+0000","last_clean":"2024-09-15T21:22:59.401472+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:59.401472+0000","last_undegraded":"2024-09-15T21:22:59.401472+0000","last_fullsized":"2024-09-15T21:22:59.401472+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11004,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11050","reported_seq":11173,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.220092+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:22:58.220092+0000","last_peered":"2024-09-15T21:22:58.220092+0000","last_clean":"2024-09-15T21:22:58.220092+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:22:58.220092+0000","last_undegraded":"2024-09-15T21:22:58.220092+0000","last_fullsized":"2024-09-15T21:22:58.220092+0000","mapping_epoch":35,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8908,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'10870","reported_seq":11328,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.378770+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:22:58.378770+0000","last_peered":"2024-09-15T21:22:58.378770+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:22:58.378770+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8295,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4302},{"shards":"0,2","objects":1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11111","reported_seq":11506,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.339576+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:22:58.339576+0000","last_peered":"2024-09-15T21:22:58.339576+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:22:58.339576+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31920128,"num_objects":7793,"num_object_clones":0,"num_object_copies":15586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8747,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4456},{"shards":"0,2","objects":1813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10872","reported_seq":10996,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.294395+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:22:58.294395+0000","last_peered":"2024-09-15T21:22:58.294395+0000","last_clean":"2024-09-15T21:22:58.294395+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:22:58.294395+0000","last_undegraded":"2024-09-15T21:22:58.294395+0000","last_fullsized":"2024-09-15T21:22:58.294395+0000","mapping_epoch":35,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30851072,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10841","reported_seq":14687,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.466403+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:59.466403+0000","last_peered":"2024-09-15T21:22:59.466403+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:59.466403+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9427,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1858,"num_bytes_recovered":7577600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1359},{"shards":"2","objects":6052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10651","reported_seq":11066,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.332928+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:22:58.332928+0000","last_peered":"2024-09-15T21:22:58.332928+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:22:58.332928+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'600","ondisk_log_start":"37'600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30240768,"num_objects":7383,"num_object_clones":0,"num_object_copies":14766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8177,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4186},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10784","reported_seq":11048,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.362309+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:59.362309+0000","last_peered":"2024-09-15T21:22:59.362309+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:59.362309+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1627},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10969","reported_seq":11220,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.373976+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:59.373976+0000","last_peered":"2024-09-15T21:22:59.373976+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:59.373976+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9368,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1658},{"shards":"2","objects":4295}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11007","reported_seq":11416,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.363948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:22:58.363948+0000","last_peered":"2024-09-15T21:22:58.363948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:22:58.363948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8314,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4408},{"shards":"0,2","objects":1782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10867","reported_seq":11235,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.255942+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:22:58.255942+0000","last_peered":"2024-09-15T21:22:58.255942+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:22:58.255942+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30904320,"num_objects":7545,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8452,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4283},{"shards":"0,2","objects":1722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10823","reported_seq":11211,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.317996+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:22:58.317996+0000","last_peered":"2024-09-15T21:22:58.317996+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:22:58.317996+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11046","reported_seq":18135,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.374509+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:22:58.374509+0000","last_peered":"2024-09-15T21:22:58.374509+0000","last_clean":"2024-09-15T21:22:58.374509+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:22:58.374509+0000","last_undegraded":"2024-09-15T21:22:58.374509+0000","last_fullsized":"2024-09-15T21:22:58.374509+0000","mapping_epoch":47,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31015529,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8706,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":497635969,"num_objects":121495,"num_object_clones":0,"num_object_copies":242990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29368,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":121495,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146284,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9838,"num_bytes_recovered":35684352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092243456,"data_stored":1092169211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609412,"kb_used_data":352820,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93762428,"statfs":{"total":96636764160,"available":96012726272,"internally_reserved":0,"allocated":361287680,"data_stored":361007318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149768,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769800,"kb_used_data":418872,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93602040,"statfs":{"total":96636764160,"available":95848488960,"internally_reserved":0,"allocated":428924928,"data_stored":428648638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,1,3,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626486,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563848,"kb_used_data":297528,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93807992,"statfs":{"total":96636764160,"available":96059383808,"internally_reserved":0,"allocated":304668672,"data_stored":304332646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":272690860},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":427917312,"data_stored":427913833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304119808,"data_stored":304056593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360206336,"data_stored":360198785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'10904", 'reported_seq': 11281, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.363632+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:22:58.363632+0000', 'last_peered': '2024-09-15T21:22:58.363632+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:22:58.363632+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31162368, 'num_objects': 7608, 'num_object_clones': 0, 'num_object_copies': 15216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3219, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10904, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 148, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4389}, {'shards': '0,2', 'objects': 1719}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'10870", 'reported_seq': 11328, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.378770+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:22:58.378770+0000', 'last_peered': '2024-09-15T21:22:58.378770+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:22:58.378770+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30765056, 'num_objects': 7511, 'num_object_clones': 0, 'num_object_copies': 15022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3211, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8295, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 162, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4302}, {'shards': '0,2', 'objects': 1694}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'11111", 'reported_seq': 11506, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.339576+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:22:58.339576+0000', 'last_peered': '2024-09-15T21:22:58.339576+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:22:58.339576+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31920128, 'num_objects': 7793, 'num_object_clones': 0, 'num_object_copies': 15586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3339, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8747, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 134, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4456}, {'shards': '0,2', 'objects': 1813}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10841", 'reported_seq': 14687, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:59.466403+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:22:59.466403+0000', 'last_peered': '2024-09-15T21:22:59.466403+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:22:59.466403+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30859264, 'num_objects': 7534, 'num_object_clones': 0, 'num_object_copies': 15068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 1359, 'num_objects_unfound': 0, 'num_objects_dirty': 7534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9427, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1858, 'num_bytes_recovered': 7577600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1359}, {'shards': '2', 'objects': 6052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:03.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10651", 'reported_seq': 11066, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.332928+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:22:58.332928+0000', 'last_peered': '2024-09-15T21:22:58.332928+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:22:58.332928+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'600", 'ondisk_log_start': "37'600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30240768, 'num_objects': 7383, 'num_object_clones': 0, 'num_object_copies': 14766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3199, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7383, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8177, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4186}, {'shards': '0,2', 'objects': 1711}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10784", 'reported_seq': 11048, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:59.362309+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:22:59.362309+0000', 'last_peered': '2024-09-15T21:22:59.362309+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:22:59.362309+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30724096, 'num_objects': 7501, 'num_object_clones': 0, 'num_object_copies': 15002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3193, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9335, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1627}, {'shards': '2', 'objects': 4310}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'10969", 'reported_seq': 11220, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:59.373976+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:22:59.373976+0000', 'last_peered': '2024-09-15T21:22:59.373976+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:22:59.373976+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31043584, 'num_objects': 7579, 'num_object_clones': 0, 'num_object_copies': 15158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3286, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7579, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9368, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1658}, {'shards': '2', 'objects': 4295}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11007", 'reported_seq': 11416, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.363948+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:22:58.363948+0000', 'last_peered': '2024-09-15T21:22:58.363948+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:22:58.363948+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31551488, 'num_objects': 7703, 'num_object_clones': 0, 'num_object_copies': 15406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3296, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8314, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 138, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4408}, {'shards': '0,2', 'objects': 1782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10867", 'reported_seq': 11235, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.255942+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:22:58.255942+0000', 'last_peered': '2024-09-15T21:22:58.255942+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:22:58.255942+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30904320, 'num_objects': 7545, 'num_object_clones': 0, 'num_object_copies': 15090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3265, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8452, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 117, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4283}, {'shards': '0,2', 'objects': 1722}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:03.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10823", 'reported_seq': 11211, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.317996+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:22:58.317996+0000', 'last_peered': '2024-09-15T21:22:58.317996+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:22:58.317996+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'800", 'ondisk_log_start': "37'800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30945280, 'num_objects': 7555, 'num_object_clones': 0, 'num_object_copies': 15110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8470, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 130, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4320}, {'shards': '0,2', 'objects': 1733}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:03.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:23:04.431+0000 7faebf7fe640 0 --2- 172.21.15.114:0/1412501269 >> [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] conn(0x7faec0093b30 0x7faec0093f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:23:05.146 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:05.146 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29237/240236 objects degraded (12.170%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":398,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120118,"data_bytes":491987604,"bytes_used":1996808192,"bytes_avail":287913484288,"bytes_total":289910292480,"degraded_objects":29237,"degraded_total":240236,"degraded_ratio":0.12170116052548327,"misplaced_objects":1359,"misplaced_total":240236,"misplaced_ratio":0.0056569373449441383,"recovering_objects_per_sec":58,"recovering_bytes_per_sec":117391,"recovering_keys_per_sec":0,"num_objects_recovered":707,"num_bytes_recovered":1409024,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:05.195 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:05.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:05.767 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:05.768 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:05.812 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-15T21:23:04.427666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491987604,"num_objects":120118,"num_object_clones":0,"num_object_copies":240236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29237,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":120118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147665,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9970,"num_bytes_recovered":35684371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1950008,"kb_used_data":1062276,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281165512,"statfs":{"total":289910292480,"available":287913484288,"internally_reserved":0,"allocated":1087770624,"data_stored":1086877946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":908971018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,7,11,3,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22769664,"num_objects":-5559,"num_object_clones":0,"num_object_copies":-11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-363,"num_objects_misplaced":-344,"num_objects_unfound":0,"num_objects_dirty":-5559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":707,"num_bytes_recovered":1409024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002803"},"pg_stats":[{"pgid":"2.f","version":"48'11019","reported_seq":11430,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.406260+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:03.406260+0000","last_peered":"2024-09-15T21:23:03.406260+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:03.406260+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11019,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4291},{"shards":"0,2","objects":1604}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11003","reported_seq":11104,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.410809+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:22:59.410809+0000","last_peered":"2024-09-15T21:22:59.410809+0000","last_clean":"2024-09-15T21:22:59.410809+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:22:59.410809+0000","last_undegraded":"2024-09-15T21:22:59.410809+0000","last_fullsized":"2024-09-15T21:22:59.410809+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31281152,"num_objects":7637,"num_object_clones":0,"num_object_copies":15274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11181","reported_seq":18101,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.357509+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:03.357509+0000","last_peered":"2024-09-15T21:23:03.357509+0000","last_clean":"2024-09-15T21:23:03.357509+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:03.357509+0000","last_undegraded":"2024-09-15T21:23:03.357509+0000","last_fullsized":"2024-09-15T21:23:03.357509+0000","mapping_epoch":45,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490072,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9904,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11004","reported_seq":11078,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:59.401472+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:22:59.401472+0000","last_peered":"2024-09-15T21:22:59.401472+0000","last_clean":"2024-09-15T21:22:59.401472+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:22:59.401472+0000","last_undegraded":"2024-09-15T21:22:59.401472+0000","last_fullsized":"2024-09-15T21:22:59.401472+0000","mapping_epoch":36,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11004,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11201","reported_seq":11324,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.428914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:03.428914+0000","last_peered":"2024-09-15T21:23:03.428914+0000","last_clean":"2024-09-15T21:23:03.428914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:03.428914+0000","last_undegraded":"2024-09-15T21:23:03.428914+0000","last_fullsized":"2024-09-15T21:23:03.428914+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11014","reported_seq":11545,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.432869+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:03.432869+0000","last_peered":"2024-09-15T21:23:03.432869+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:03.432869+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8439,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11233","reported_seq":11649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.289889+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:03.289889+0000","last_peered":"2024-09-15T21:23:03.289889+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:03.289889+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420416,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8869,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4344},{"shards":"0,2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10992","reported_seq":11116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.222525+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:03.222525+0000","last_peered":"2024-09-15T21:23:03.222525+0000","last_clean":"2024-09-15T21:23:03.222525+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:03.222525+0000","last_undegraded":"2024-09-15T21:23:03.222525+0000","last_fullsized":"2024-09-15T21:23:03.222525+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8777,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10841","reported_seq":14687,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.466403+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:22:59.466403+0000","last_peered":"2024-09-15T21:22:59.466403+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:22:59.466403+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'800","ondisk_log_start":"37'800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9427,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1858,"num_bytes_recovered":7577600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1359},{"shards":"2","objects":6052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10762","reported_seq":11227,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.396786+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:03.396786+0000","last_peered":"2024-09-15T21:23:03.396786+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:03.396786+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4100},{"shards":"0,2","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10784","reported_seq":11048,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.362309+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:22:59.362309+0000","last_peered":"2024-09-15T21:22:59.362309+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:22:59.362309+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30724096,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1627},{"shards":"2","objects":4310}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'10969","reported_seq":11220,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:59.373976+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:22:59.373976+0000","last_peered":"2024-09-15T21:22:59.373976+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:22:59.373976+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9368,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1658},{"shards":"2","objects":4295}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11113","reported_seq":11544,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.355822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:03.355822+0000","last_peered":"2024-09-15T21:23:03.355822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:03.355822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8420,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10990","reported_seq":11386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.357755+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:03.357755+0000","last_peered":"2024-09-15T21:23:03.357755+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:03.357755+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8575,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10971","reported_seq":11393,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.385713+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:03.385713+0000","last_peered":"2024-09-15T21:23:03.385713+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:03.385713+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11172","reported_seq":18261,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.361854+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:03.361854+0000","last_peered":"2024-09-15T21:23:03.361854+0000","last_clean":"2024-09-15T21:23:03.361854+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:03.361854+0000","last_undegraded":"2024-09-15T21:23:03.361854+0000","last_fullsized":"2024-09-15T21:23:03.361854+0000","mapping_epoch":47,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30499433,"num_objects":7447,"num_object_clones":0,"num_object_copies":14894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8832,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491987585,"num_objects":120116,"num_object_clones":0,"num_object_copies":240232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29237,"num_objects_misplaced":1359,"num_objects_unfound":0,"num_objects_dirty":120116,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147663,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9969,"num_bytes_recovered":35684352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085206528,"data_stored":1085132283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953541,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609412,"kb_used_data":352820,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93762428,"statfs":{"total":96636764160,"available":96012726272,"internally_reserved":0,"allocated":361287680,"data_stored":361007318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751108,"kb_used_data":412656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93620732,"statfs":{"total":96636764160,"available":95867629568,"internally_reserved":0,"allocated":422559744,"data_stored":422283454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,1,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589488,"kb_used_data":296800,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93782352,"statfs":{"total":96636764160,"available":96033128448,"internally_reserved":0,"allocated":303923200,"data_stored":303587174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421523456,"data_stored":421519977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303476736,"data_stored":303413521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360206336,"data_stored":360198785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:08.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:09.688 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:09.690 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:09.730 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-15T21:23:08.428662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489259668,"num_objects":119452,"num_object_clones":0,"num_object_copies":238904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29227,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":119452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148331,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10167,"num_bytes_recovered":36450323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1932660,"kb_used_data":1057408,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281182860,"statfs":{"total":289910292480,"available":287931248640,"internally_reserved":0,"allocated":1082785792,"data_stored":1081893114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,7,11,3,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17416192,"num_objects":-4252,"num_object_clones":0,"num_object_copies":-8504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-226,"num_objects_misplaced":-354,"num_objects_unfound":0,"num_objects_dirty":-4252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4252,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"2.f","version":"48'11019","reported_seq":11430,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.406260+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:03.406260+0000","last_peered":"2024-09-15T21:23:03.406260+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:03.406260+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11019,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4291},{"shards":"0,2","objects":1604}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11144","reported_seq":11242,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:04.431681+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:04.431681+0000","last_peered":"2024-09-15T21:23:04.431681+0000","last_clean":"2024-09-15T21:23:04.431681+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:04.431681+0000","last_undegraded":"2024-09-15T21:23:04.431681+0000","last_fullsized":"2024-09-15T21:23:04.431681+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30703616,"num_objects":7496,"num_object_clones":0,"num_object_copies":14992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9870,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11181","reported_seq":18101,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.357509+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:03.357509+0000","last_peered":"2024-09-15T21:23:03.357509+0000","last_clean":"2024-09-15T21:23:03.357509+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:03.357509+0000","last_undegraded":"2024-09-15T21:23:03.357509+0000","last_fullsized":"2024-09-15T21:23:03.357509+0000","mapping_epoch":45,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490072,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9904,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11159","reported_seq":11233,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:04.480979+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:04.480979+0000","last_peered":"2024-09-15T21:23:04.480979+0000","last_clean":"2024-09-15T21:23:04.480979+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:04.480979+0000","last_undegraded":"2024-09-15T21:23:04.480979+0000","last_fullsized":"2024-09-15T21:23:04.480979+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30355456,"num_objects":7411,"num_object_clones":0,"num_object_copies":14822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11159,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11201","reported_seq":11324,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.428914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:03.428914+0000","last_peered":"2024-09-15T21:23:03.428914+0000","last_clean":"2024-09-15T21:23:03.428914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:03.428914+0000","last_undegraded":"2024-09-15T21:23:03.428914+0000","last_fullsized":"2024-09-15T21:23:03.428914+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11014","reported_seq":11545,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.432869+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:03.432869+0000","last_peered":"2024-09-15T21:23:03.432869+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:03.432869+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8439,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11233","reported_seq":11649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.289889+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:03.289889+0000","last_peered":"2024-09-15T21:23:03.289889+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:03.289889+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420416,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8869,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4344},{"shards":"0,2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10992","reported_seq":11116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.222525+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:03.222525+0000","last_peered":"2024-09-15T21:23:03.222525+0000","last_clean":"2024-09-15T21:23:03.222525+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:03.222525+0000","last_undegraded":"2024-09-15T21:23:03.222525+0000","last_fullsized":"2024-09-15T21:23:03.222525+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8777,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10961","reported_seq":15181,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.485037+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:04.485037+0000","last_peered":"2024-09-15T21:23:04.485037+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:04.485037+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9547,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2045,"num_bytes_recovered":8343552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1172},{"shards":"2","objects":6119}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10762","reported_seq":11227,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.396786+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:03.396786+0000","last_peered":"2024-09-15T21:23:03.396786+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:03.396786+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4100},{"shards":"0,2","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10906","reported_seq":11176,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.413991+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:04.413991+0000","last_peered":"2024-09-15T21:23:04.413991+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:04.413991+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9457,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1505},{"shards":"2","objects":4191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11097","reported_seq":11362,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.444231+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:04.444231+0000","last_peered":"2024-09-15T21:23:04.444231+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:04.444231+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519296,"num_objects":7451,"num_object_clones":0,"num_object_copies":14902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9496,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1530},{"shards":"2","objects":4174}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11113","reported_seq":11544,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.355822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:03.355822+0000","last_peered":"2024-09-15T21:23:03.355822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:03.355822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8420,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10990","reported_seq":11386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.357755+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:03.357755+0000","last_peered":"2024-09-15T21:23:03.357755+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:03.357755+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8575,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10971","reported_seq":11393,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.385713+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:03.385713+0000","last_peered":"2024-09-15T21:23:03.385713+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:03.385713+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11172","reported_seq":18261,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.361854+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:03.361854+0000","last_peered":"2024-09-15T21:23:03.361854+0000","last_clean":"2024-09-15T21:23:03.361854+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:03.361854+0000","last_undegraded":"2024-09-15T21:23:03.361854+0000","last_fullsized":"2024-09-15T21:23:03.361854+0000","mapping_epoch":47,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30499433,"num_objects":7447,"num_object_clones":0,"num_object_copies":14894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8832,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489259649,"num_objects":119450,"num_object_clones":0,"num_object_copies":238900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29227,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":119450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148329,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10166,"num_bytes_recovered":36450304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1080856576,"data_stored":1080782331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592064,"kb_used_data":347952,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93779776,"statfs":{"total":96636764160,"available":96030490624,"internally_reserved":0,"allocated":356302848,"data_stored":356022486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751108,"kb_used_data":412656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93620732,"statfs":{"total":96636764160,"available":95867629568,"internally_reserved":0,"allocated":422559744,"data_stored":422283454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,1,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589488,"kb_used_data":296800,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93782352,"statfs":{"total":96636764160,"available":96033128448,"internally_reserved":0,"allocated":303923200,"data_stored":303587174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421523456,"data_stored":421519977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303476736,"data_stored":303413521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355856384,"data_stored":355848833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:09.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:10.328 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:10.330 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:10.361 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-15T21:23:08.428662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489259668,"num_objects":119452,"num_object_clones":0,"num_object_copies":238904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29227,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":119452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148331,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10167,"num_bytes_recovered":36450323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1932660,"kb_used_data":1057408,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281182860,"statfs":{"total":289910292480,"available":287931248640,"internally_reserved":0,"allocated":1082785792,"data_stored":1081893114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,7,11,3,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17416192,"num_objects":-4252,"num_object_clones":0,"num_object_copies":-8504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-226,"num_objects_misplaced":-354,"num_objects_unfound":0,"num_objects_dirty":-4252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4252,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"2.f","version":"48'11019","reported_seq":11430,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.406260+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:03.406260+0000","last_peered":"2024-09-15T21:23:03.406260+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:03.406260+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11019,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4291},{"shards":"0,2","objects":1604}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11144","reported_seq":11242,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:04.431681+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:04.431681+0000","last_peered":"2024-09-15T21:23:04.431681+0000","last_clean":"2024-09-15T21:23:04.431681+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:04.431681+0000","last_undegraded":"2024-09-15T21:23:04.431681+0000","last_fullsized":"2024-09-15T21:23:04.431681+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30703616,"num_objects":7496,"num_object_clones":0,"num_object_copies":14992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9870,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11181","reported_seq":18101,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.357509+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:03.357509+0000","last_peered":"2024-09-15T21:23:03.357509+0000","last_clean":"2024-09-15T21:23:03.357509+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:03.357509+0000","last_undegraded":"2024-09-15T21:23:03.357509+0000","last_fullsized":"2024-09-15T21:23:03.357509+0000","mapping_epoch":45,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490072,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9904,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11159","reported_seq":11233,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:04.480979+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:04.480979+0000","last_peered":"2024-09-15T21:23:04.480979+0000","last_clean":"2024-09-15T21:23:04.480979+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:04.480979+0000","last_undegraded":"2024-09-15T21:23:04.480979+0000","last_fullsized":"2024-09-15T21:23:04.480979+0000","mapping_epoch":36,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30355456,"num_objects":7411,"num_object_clones":0,"num_object_copies":14822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11159,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11201","reported_seq":11324,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.428914+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:03.428914+0000","last_peered":"2024-09-15T21:23:03.428914+0000","last_clean":"2024-09-15T21:23:03.428914+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:03.428914+0000","last_undegraded":"2024-09-15T21:23:03.428914+0000","last_fullsized":"2024-09-15T21:23:03.428914+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11014","reported_seq":11545,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.432869+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:03.432869+0000","last_peered":"2024-09-15T21:23:03.432869+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:03.432869+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8439,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":1550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11233","reported_seq":11649,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.289889+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:03.289889+0000","last_peered":"2024-09-15T21:23:03.289889+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:03.289889+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420416,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8869,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4344},{"shards":"0,2","objects":1691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'10992","reported_seq":11116,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.222525+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:03.222525+0000","last_peered":"2024-09-15T21:23:03.222525+0000","last_clean":"2024-09-15T21:23:03.222525+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:03.222525+0000","last_undegraded":"2024-09-15T21:23:03.222525+0000","last_fullsized":"2024-09-15T21:23:03.222525+0000","mapping_epoch":35,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8777,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'10961","reported_seq":15181,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.485037+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:04.485037+0000","last_peered":"2024-09-15T21:23:04.485037+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:04.485037+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9547,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2045,"num_bytes_recovered":8343552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1172},{"shards":"2","objects":6119}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10762","reported_seq":11227,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.396786+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:03.396786+0000","last_peered":"2024-09-15T21:23:03.396786+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:03.396786+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'700","ondisk_log_start":"37'700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4100},{"shards":"0,2","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'10906","reported_seq":11176,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.413991+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:04.413991+0000","last_peered":"2024-09-15T21:23:04.413991+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:04.413991+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9457,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1505},{"shards":"2","objects":4191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11097","reported_seq":11362,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:04.444231+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:04.444231+0000","last_peered":"2024-09-15T21:23:04.444231+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:04.444231+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30519296,"num_objects":7451,"num_object_clones":0,"num_object_copies":14902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9496,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1530},{"shards":"2","objects":4174}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11113","reported_seq":11544,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.355822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:03.355822+0000","last_peered":"2024-09-15T21:23:03.355822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:03.355822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8420,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'10990","reported_seq":11386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.357755+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:03.357755+0000","last_peered":"2024-09-15T21:23:03.357755+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:03.357755+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30400512,"num_objects":7422,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8575,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4174},{"shards":"0,2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'10971","reported_seq":11393,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.385713+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:03.385713+0000","last_peered":"2024-09-15T21:23:03.385713+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:03.385713+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4189},{"shards":"0,2","objects":1585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11172","reported_seq":18261,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.361854+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:03.361854+0000","last_peered":"2024-09-15T21:23:03.361854+0000","last_clean":"2024-09-15T21:23:03.361854+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:03.361854+0000","last_undegraded":"2024-09-15T21:23:03.361854+0000","last_fullsized":"2024-09-15T21:23:03.361854+0000","mapping_epoch":47,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30499433,"num_objects":7447,"num_object_clones":0,"num_object_copies":14894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8832,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489259649,"num_objects":119450,"num_object_clones":0,"num_object_copies":238900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29227,"num_objects_misplaced":1172,"num_objects_unfound":0,"num_objects_dirty":119450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148329,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10166,"num_bytes_recovered":36450304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1080856576,"data_stored":1080782331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953542,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592064,"kb_used_data":347952,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93779776,"statfs":{"total":96636764160,"available":96030490624,"internally_reserved":0,"allocated":356302848,"data_stored":356022486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149769,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751108,"kb_used_data":412656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93620732,"statfs":{"total":96636764160,"available":95867629568,"internally_reserved":0,"allocated":422559744,"data_stored":422283454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,1,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589488,"kb_used_data":296800,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93782352,"statfs":{"total":96636764160,"available":96033128448,"internally_reserved":0,"allocated":303923200,"data_stored":303587174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421523456,"data_stored":421519977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303476736,"data_stored":303413521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355856384,"data_stored":355848833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:10.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:10.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11019", 'reported_seq': 11430, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.406260+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:03.406260+0000', 'last_peered': '2024-09-15T21:23:03.406260+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:03.406260+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30691328, 'num_objects': 7493, 'num_object_clones': 0, 'num_object_copies': 14986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3202, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11019, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 165, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4291}, {'shards': '0,2', 'objects': 1604}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:10.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11014", 'reported_seq': 11545, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.432869+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:03.432869+0000', 'last_peered': '2024-09-15T21:23:03.432869+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:03.432869+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30175232, 'num_objects': 7367, 'num_object_clones': 0, 'num_object_copies': 14734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8439, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 199, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4195}, {'shards': '0,2', 'objects': 1550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'11233", 'reported_seq': 11649, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.289889+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:03.289889+0000', 'last_peered': '2024-09-15T21:23:03.289889+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:03.289889+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31420416, 'num_objects': 7671, 'num_object_clones': 0, 'num_object_copies': 15342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3329, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8869, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4344}, {'shards': '0,2', 'objects': 1691}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'10961", 'reported_seq': 15181, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:04.485037+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:23:04.485037+0000', 'last_peered': '2024-09-15T21:23:04.485037+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:04.485037+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30367744, 'num_objects': 7414, 'num_object_clones': 0, 'num_object_copies': 14828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 1172, 'num_objects_unfound': 0, 'num_objects_dirty': 7414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9547, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2045, 'num_bytes_recovered': 8343552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1172}, {'shards': '2', 'objects': 6119}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'10762", 'reported_seq': 11227, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.396786+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:03.396786+0000', 'last_peered': '2024-09-15T21:23:03.396786+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:03.396786+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'700", 'ondisk_log_start': "37'700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29786112, 'num_objects': 7272, 'num_object_clones': 0, 'num_object_copies': 14544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8288, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 169, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4100}, {'shards': '0,2', 'objects': 1600}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'10906", 'reported_seq': 11176, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:04.413991+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:04.413991+0000', 'last_peered': '2024-09-15T21:23:04.413991+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:04.413991+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30224384, 'num_objects': 7379, 'num_object_clones': 0, 'num_object_copies': 14758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9457, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 78, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1505}, {'shards': '2', 'objects': 4191}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:10.363 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'11097", 'reported_seq': 11362, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:04.444231+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:04.444231+0000', 'last_peered': '2024-09-15T21:23:04.444231+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:04.444231+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30519296, 'num_objects': 7451, 'num_object_clones': 0, 'num_object_copies': 14902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3279, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9496, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 73, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1530}, {'shards': '2', 'objects': 4174}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11113", 'reported_seq': 11544, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.355822+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:03.355822+0000', 'last_peered': '2024-09-15T21:23:03.355822+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:03.355822+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31117312, 'num_objects': 7597, 'num_object_clones': 0, 'num_object_copies': 15194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8420, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 149, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4313}, {'shards': '0,2', 'objects': 1676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'10990", 'reported_seq': 11386, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.357755+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:23:03.357755+0000', 'last_peered': '2024-09-15T21:23:03.357755+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:03.357755+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30400512, 'num_objects': 7422, 'num_object_clones': 0, 'num_object_copies': 14844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3251, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8575, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4174}, {'shards': '0,2', 'objects': 1599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:10.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'10971", 'reported_seq': 11393, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.385713+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:03.385713+0000', 'last_peered': '2024-09-15T21:23:03.385713+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:03.385713+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'900", 'ondisk_log_start': "37'900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30339072, 'num_objects': 7407, 'num_object_clones': 0, 'num_object_copies': 14814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3220, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8618, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 147, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4189}, {'shards': '0,2', 'objects': 1585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:10.364 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:11.368 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:11.368 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29131/234520 objects degraded (12.422%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":404,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":117260,"data_bytes":480281236,"bytes_used":1967316992,"bytes_avail":287942975488,"bytes_total":289910292480,"degraded_objects":29131,"degraded_total":234520,"degraded_ratio":0.12421541872761384,"misplaced_objects":969,"misplaced_total":234520,"misplaced_ratio":0.0041318437659901078,"recovering_objects_per_sec":66,"recovering_bytes_per_sec":190078,"recovering_keys_per_sec":0,"num_objects_recovered":803,"num_bytes_recovered":2281472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":432},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:11.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:11.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:12.141 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:12.188 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-15T21:23:10.429083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480281236,"num_objects":117260,"num_object_clones":0,"num_object_copies":234520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29131,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":117260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150523,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10466,"num_bytes_recovered":37281811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1921208,"kb_used_data":1045948,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281194312,"statfs":{"total":289910292480,"available":287942975488,"internally_reserved":0,"allocated":1071050752,"data_stored":1070158074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,10,8,2,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21266432,"num_objects":-5192,"num_object_clones":0,"num_object_copies":-10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":-557,"num_objects_unfound":0,"num_objects_dirty":-5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5192,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":2281472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002811"},"pg_stats":[{"pgid":"2.f","version":"48'11171","reported_seq":11602,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.426963+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:08.426963+0000","last_peered":"2024-09-15T21:23:08.426963+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:08.426963+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30068736,"num_objects":7341,"num_object_clones":0,"num_object_copies":14682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11171,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4149},{"shards":"0,2","objects":1452}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11294","reported_seq":11395,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:09.469351+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:09.469351+0000","last_peered":"2024-09-15T21:23:09.469351+0000","last_clean":"2024-09-15T21:23:09.469351+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:09.469351+0000","last_undegraded":"2024-09-15T21:23:09.469351+0000","last_fullsized":"2024-09-15T21:23:09.469351+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30089216,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11338","reported_seq":18259,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.497579+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:08.497579+0000","last_peered":"2024-09-15T21:23:08.497579+0000","last_clean":"2024-09-15T21:23:08.497579+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:08.497579+0000","last_undegraded":"2024-09-15T21:23:08.497579+0000","last_fullsized":"2024-09-15T21:23:08.497579+0000","mapping_epoch":45,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30847000,"num_objects":7532,"num_object_clones":0,"num_object_copies":15064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10061,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11277","reported_seq":11351,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:09.482605+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:09.482605+0000","last_peered":"2024-09-15T21:23:09.482605+0000","last_clean":"2024-09-15T21:23:09.482605+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:09.482605+0000","last_undegraded":"2024-09-15T21:23:09.482605+0000","last_fullsized":"2024-09-15T21:23:09.482605+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29872128,"num_objects":7293,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11277,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11330","reported_seq":11452,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.438275+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:08.438275+0000","last_peered":"2024-09-15T21:23:08.438275+0000","last_clean":"2024-09-15T21:23:08.438275+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:08.438275+0000","last_undegraded":"2024-09-15T21:23:08.438275+0000","last_fullsized":"2024-09-15T21:23:08.438275+0000","mapping_epoch":35,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30281728,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11133","reported_seq":11695,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.411946+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:08.411946+0000","last_peered":"2024-09-15T21:23:08.411946+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:08.411946+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8558,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":1431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11385","reported_seq":11827,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.401511+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:08.401511+0000","last_peered":"2024-09-15T21:23:08.401511+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:08.401511+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9021,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4205},{"shards":"0,2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11132","reported_seq":11256,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.439014+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:08.439014+0000","last_peered":"2024-09-15T21:23:08.439014+0000","last_clean":"2024-09-15T21:23:08.439014+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:08.439014+0000","last_undegraded":"2024-09-15T21:23:08.439014+0000","last_fullsized":"2024-09-15T21:23:08.439014+0000","mapping_epoch":35,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29786112,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8917,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11078","reported_seq":15704,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.457030+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:09.457030+0000","last_peered":"2024-09-15T21:23:09.457030+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:09.457030+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9664,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":9175040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":969},{"shards":"2","objects":6205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'10902","reported_seq":11386,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.395734+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:08.395734+0000","last_peered":"2024-09-15T21:23:08.395734+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:08.395734+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'900","ondisk_log_start":"37'900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29212672,"num_objects":7132,"num_object_clones":0,"num_object_copies":14264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8428,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":1460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11040","reported_seq":11328,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.450220+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:09.450220+0000","last_peered":"2024-09-15T21:23:09.450220+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:09.450220+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675520,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1371},{"shards":"2","objects":4066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11239","reported_seq":11522,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.449968+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:09.449968+0000","last_peered":"2024-09-15T21:23:09.449968+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:09.449968+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29937664,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9638,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1388},{"shards":"2","objects":4041}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11279","reported_seq":11742,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.426221+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:08.426221+0000","last_peered":"2024-09-15T21:23:08.426221+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:08.426221+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8586,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4163},{"shards":"0,2","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11109","reported_seq":11517,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.390571+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:08.390571+0000","last_peered":"2024-09-15T21:23:08.390571+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:08.390571+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913088,"num_objects":7303,"num_object_clones":0,"num_object_copies":14606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8694,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4061},{"shards":"0,2","objects":1480}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11096","reported_seq":11534,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.390822+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:08.390822+0000","last_peered":"2024-09-15T21:23:08.390822+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:08.390822+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29827072,"num_objects":7282,"num_object_clones":0,"num_object_copies":14564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4072},{"shards":"0,2","objects":1460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11304","reported_seq":18391,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.478410+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:08.478410+0000","last_peered":"2024-09-15T21:23:08.478410+0000","last_clean":"2024-09-15T21:23:08.478410+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:08.478410+0000","last_undegraded":"2024-09-15T21:23:08.478410+0000","last_fullsized":"2024-09-15T21:23:08.478410+0000","mapping_epoch":47,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29958761,"num_objects":7315,"num_object_clones":0,"num_object_copies":14630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8964,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480281217,"num_objects":117258,"num_object_clones":0,"num_object_copies":234516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29131,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":117258,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150521,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10465,"num_bytes_recovered":37281792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068994560,"data_stored":1068920315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591912,"kb_used_data":343632,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93779928,"statfs":{"total":96636764160,"available":96030646272,"internally_reserved":0,"allocated":351879168,"data_stored":351598806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149770,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748712,"kb_used_data":406104,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93623128,"statfs":{"total":96636764160,"available":95870083072,"internally_reserved":0,"allocated":415850496,"data_stored":415574206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2,0,0,0,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580584,"kb_used_data":296212,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93791256,"statfs":{"total":96636764160,"available":96042246144,"internally_reserved":0,"allocated":303321088,"data_stored":302985062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":414859264,"data_stored":414855785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302751744,"data_stored":302688529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351383552,"data_stored":351376001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:15.190 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:15.836 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:15.837 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:15.896 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-15T21:23:14.429998+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474374804,"num_objects":115818,"num_object_clones":0,"num_object_copies":231636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29056,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":115818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151965,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10541,"num_bytes_recovered":37281811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926448,"kb_used_data":1038704,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281189072,"statfs":{"total":289910292480,"available":287937609728,"internally_reserved":0,"allocated":1063632896,"data_stored":1062740218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":908971018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,8,13,4,1,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23261184,"num_objects":-5679,"num_object_clones":0,"num_object_copies":-11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":-390,"num_objects_unfound":0,"num_objects_dirty":-5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5679,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":702,"num_bytes_recovered":1597440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002768"},"pg_stats":[{"pgid":"2.f","version":"48'11290","reported_seq":11725,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.411129+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:13.411129+0000","last_peered":"2024-09-15T21:23:13.411129+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:13.411129+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11290,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4032},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11294","reported_seq":11395,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:09.469351+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:09.469351+0000","last_peered":"2024-09-15T21:23:09.469351+0000","last_clean":"2024-09-15T21:23:09.469351+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:09.469351+0000","last_undegraded":"2024-09-15T21:23:09.469351+0000","last_fullsized":"2024-09-15T21:23:09.469351+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30089216,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11471","reported_seq":18392,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.507926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:13.507926+0000","last_peered":"2024-09-15T21:23:13.507926+0000","last_clean":"2024-09-15T21:23:13.507926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:13.507926+0000","last_undegraded":"2024-09-15T21:23:13.507926+0000","last_fullsized":"2024-09-15T21:23:13.507926+0000","mapping_epoch":45,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30302232,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10194,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11277","reported_seq":11351,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:09.482605+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:09.482605+0000","last_peered":"2024-09-15T21:23:09.482605+0000","last_clean":"2024-09-15T21:23:09.482605+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:09.482605+0000","last_undegraded":"2024-09-15T21:23:09.482605+0000","last_fullsized":"2024-09-15T21:23:09.482605+0000","mapping_epoch":36,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29872128,"num_objects":7293,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11277,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11456","reported_seq":11579,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.380253+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:13.380253+0000","last_peered":"2024-09-15T21:23:13.380253+0000","last_clean":"2024-09-15T21:23:13.380253+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:13.380253+0000","last_undegraded":"2024-09-15T21:23:13.380253+0000","last_fullsized":"2024-09-15T21:23:13.380253+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29765632,"num_objects":7267,"num_object_clones":0,"num_object_copies":14534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9314,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11254","reported_seq":11846,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.399808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:13.399808+0000","last_peered":"2024-09-15T21:23:13.399808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:13.399808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8679,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3985},{"shards":"0,2","objects":1310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11512","reported_seq":11978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418803+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:13.418803+0000","last_peered":"2024-09-15T21:23:13.418803+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:13.418803+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4090},{"shards":"0,2","objects":1412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11293","reported_seq":11416,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.390383+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:13.390383+0000","last_peered":"2024-09-15T21:23:13.390383+0000","last_clean":"2024-09-15T21:23:13.390383+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:13.390383+0000","last_undegraded":"2024-09-15T21:23:13.390383+0000","last_fullsized":"2024-09-15T21:23:13.390383+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29126656,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9078,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11078","reported_seq":15704,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.457030+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:09.457030+0000","last_peered":"2024-09-15T21:23:09.457030+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:09.457030+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9664,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":9175040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":969},{"shards":"2","objects":6205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11020","reported_seq":11535,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:13.418911+0000","last_peered":"2024-09-15T21:23:13.418911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:13.418911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11040","reported_seq":11328,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.450220+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:09.450220+0000","last_peered":"2024-09-15T21:23:09.450220+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:09.450220+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675520,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1371},{"shards":"2","objects":4066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11239","reported_seq":11522,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:09.449968+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:09.449968+0000","last_peered":"2024-09-15T21:23:09.449968+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:09.449968+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29937664,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9638,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1388},{"shards":"2","objects":4041}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11412","reported_seq":11887,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.435382+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:13.435382+0000","last_peered":"2024-09-15T21:23:13.435382+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:13.435382+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8719,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4036},{"shards":"0,2","objects":1377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11252","reported_seq":11690,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.416217+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:13.416217+0000","last_peered":"2024-09-15T21:23:13.416217+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:13.416217+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29327360,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8837,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":1337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11225","reported_seq":11683,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.349983+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:13.349983+0000","last_peered":"2024-09-15T21:23:13.349983+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:13.349983+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29298688,"num_objects":7153,"num_object_clones":0,"num_object_copies":14306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8872,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3953},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11436","reported_seq":18525,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.483369+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:13.483369+0000","last_peered":"2024-09-15T21:23:13.483369+0000","last_clean":"2024-09-15T21:23:13.483369+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:13.483369+0000","last_undegraded":"2024-09-15T21:23:13.483369+0000","last_fullsized":"2024-09-15T21:23:13.483369+0000","mapping_epoch":47,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29418089,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474374785,"num_objects":115816,"num_object_clones":0,"num_object_copies":231632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29056,"num_objects_misplaced":969,"num_objects_unfound":0,"num_objects_dirty":115816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151963,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10540,"num_bytes_recovered":37281792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061572608,"data_stored":1061498363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953543,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591912,"kb_used_data":343632,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93779928,"statfs":{"total":96636764160,"available":96030646272,"internally_reserved":0,"allocated":351879168,"data_stored":351598806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750628,"kb_used_data":399692,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93621212,"statfs":{"total":96636764160,"available":95868121088,"internally_reserved":0,"allocated":409284608,"data_stored":409008318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583908,"kb_used_data":295380,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93787932,"statfs":{"total":96636764160,"available":96038842368,"internally_reserved":0,"allocated":302469120,"data_stored":302133094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408301568,"data_stored":408298089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301887488,"data_stored":301824273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351383552,"data_stored":351376001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:15.899 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:16.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:16.671 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:16.719 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T21:23:16.430503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471548564,"num_objects":115128,"num_object_clones":0,"num_object_copies":230256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29050,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":115128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152655,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10753,"num_bytes_recovered":38125587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926276,"kb_used_data":1034372,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":281189244,"statfs":{"total":289910292480,"available":287937785856,"internally_reserved":0,"allocated":1059196928,"data_stored":1058304250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":913230858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,6,11,2,1,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20439040,"num_objects":-4990,"num_object_clones":0,"num_object_copies":-9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":-596,"num_objects_unfound":0,"num_objects_dirty":-4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4990,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":2441216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"2.f","version":"48'11290","reported_seq":11725,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.411129+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:13.411129+0000","last_peered":"2024-09-15T21:23:13.411129+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:13.411129+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11290,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4032},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11428","reported_seq":11529,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:14.432382+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:14.432382+0000","last_peered":"2024-09-15T21:23:14.432382+0000","last_clean":"2024-09-15T21:23:14.432382+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:14.432382+0000","last_undegraded":"2024-09-15T21:23:14.432382+0000","last_fullsized":"2024-09-15T21:23:14.432382+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29540352,"num_objects":7212,"num_object_clones":0,"num_object_copies":14424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10154,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11471","reported_seq":18392,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.507926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:13.507926+0000","last_peered":"2024-09-15T21:23:13.507926+0000","last_clean":"2024-09-15T21:23:13.507926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:13.507926+0000","last_undegraded":"2024-09-15T21:23:13.507926+0000","last_fullsized":"2024-09-15T21:23:13.507926+0000","mapping_epoch":45,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30302232,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10194,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11437","reported_seq":11511,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:14.422975+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:14.422975+0000","last_peered":"2024-09-15T21:23:14.422975+0000","last_clean":"2024-09-15T21:23:14.422975+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:14.422975+0000","last_undegraded":"2024-09-15T21:23:14.422975+0000","last_fullsized":"2024-09-15T21:23:14.422975+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11456","reported_seq":11579,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.380253+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:13.380253+0000","last_peered":"2024-09-15T21:23:13.380253+0000","last_clean":"2024-09-15T21:23:13.380253+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:13.380253+0000","last_undegraded":"2024-09-15T21:23:13.380253+0000","last_fullsized":"2024-09-15T21:23:13.380253+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29765632,"num_objects":7267,"num_object_clones":0,"num_object_copies":14534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9314,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11254","reported_seq":11846,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.399808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:13.399808+0000","last_peered":"2024-09-15T21:23:13.399808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:13.399808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8679,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3985},{"shards":"0,2","objects":1310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11512","reported_seq":11978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418803+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:13.418803+0000","last_peered":"2024-09-15T21:23:13.418803+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:13.418803+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4090},{"shards":"0,2","objects":1412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11293","reported_seq":11416,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.390383+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:13.390383+0000","last_peered":"2024-09-15T21:23:13.390383+0000","last_clean":"2024-09-15T21:23:13.390383+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:13.390383+0000","last_undegraded":"2024-09-15T21:23:13.390383+0000","last_fullsized":"2024-09-15T21:23:13.390383+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29126656,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9078,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11222","reported_seq":16260,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.500699+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:14.500699+0000","last_peered":"2024-09-15T21:23:14.500699+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:14.500699+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29298688,"num_objects":7153,"num_object_clones":0,"num_object_copies":14306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":7153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9808,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2454,"num_bytes_recovered":10018816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":763},{"shards":"2","objects":6267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11020","reported_seq":11535,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:13.418911+0000","last_peered":"2024-09-15T21:23:13.418911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:13.418911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11172","reported_seq":11468,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.461459+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:14.461459+0000","last_peered":"2024-09-15T21:23:14.461459+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:14.461459+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9723,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1239},{"shards":"2","objects":3938}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11359","reported_seq":11646,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.479438+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:14.479438+0000","last_peered":"2024-09-15T21:23:14.479438+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:14.479438+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9758,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1268},{"shards":"2","objects":3923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11412","reported_seq":11887,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.435382+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:13.435382+0000","last_peered":"2024-09-15T21:23:13.435382+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:13.435382+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8719,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4036},{"shards":"0,2","objects":1377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11252","reported_seq":11690,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.416217+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:13.416217+0000","last_peered":"2024-09-15T21:23:13.416217+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:13.416217+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29327360,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8837,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":1337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11225","reported_seq":11683,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.349983+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:13.349983+0000","last_peered":"2024-09-15T21:23:13.349983+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:13.349983+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29298688,"num_objects":7153,"num_object_clones":0,"num_object_copies":14306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8872,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3953},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11436","reported_seq":18525,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.483369+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:13.483369+0000","last_peered":"2024-09-15T21:23:13.483369+0000","last_clean":"2024-09-15T21:23:13.483369+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:13.483369+0000","last_undegraded":"2024-09-15T21:23:13.483369+0000","last_fullsized":"2024-09-15T21:23:13.483369+0000","mapping_epoch":47,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29418089,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471548545,"num_objects":115126,"num_object_clones":0,"num_object_copies":230252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29050,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":115126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152653,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10752,"num_bytes_recovered":38125568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057058816,"data_stored":1056984571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953544,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591740,"kb_used_data":339300,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93780100,"statfs":{"total":96636764160,"available":96030822400,"internally_reserved":0,"allocated":347443200,"data_stored":347162838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750628,"kb_used_data":399692,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93621212,"statfs":{"total":96636764160,"available":95868121088,"internally_reserved":0,"allocated":409284608,"data_stored":409008318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583908,"kb_used_data":295380,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93787932,"statfs":{"total":96636764160,"available":96038842368,"internally_reserved":0,"allocated":302469120,"data_stored":302133094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408301568,"data_stored":408298089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301887488,"data_stored":301824273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346869760,"data_stored":346862209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11290", 'reported_seq': 11725, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.411129+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:13.411129+0000', 'last_peered': '2024-09-15T21:23:13.411129+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:13.411129+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29581312, 'num_objects': 7222, 'num_object_clones': 0, 'num_object_copies': 14444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11290, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 177, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4032}, {'shards': '0,2', 'objects': 1333}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11254", 'reported_seq': 11846, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.399808+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:13.399808+0000', 'last_peered': '2024-09-15T21:23:13.399808+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:13.399808+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29192192, 'num_objects': 7127, 'num_object_clones': 0, 'num_object_copies': 14254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3144, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8679, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 229, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3985}, {'shards': '0,2', 'objects': 1310}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'11512", 'reported_seq': 11978, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.418803+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:13.418803+0000', 'last_peered': '2024-09-15T21:23:13.418803+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:13.418803+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30277632, 'num_objects': 7392, 'num_object_clones': 0, 'num_object_copies': 14784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3304, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9148, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 169, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4090}, {'shards': '0,2', 'objects': 1412}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:16.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'11222", 'reported_seq': 16260, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:14.500699+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:23:14.500699+0000', 'last_peered': '2024-09-15T21:23:14.500699+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:14.500699+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29298688, 'num_objects': 7153, 'num_object_clones': 0, 'num_object_copies': 14306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 763, 'num_objects_unfound': 0, 'num_objects_dirty': 7153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9808, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2454, 'num_bytes_recovered': 10018816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 763}, {'shards': '2', 'objects': 6267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'11020", 'reported_seq': 11535, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.418911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:13.418911+0000', 'last_peered': '2024-09-15T21:23:13.418911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:13.418911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1000", 'ondisk_log_start': "37'1000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28729344, 'num_objects': 7014, 'num_object_clones': 0, 'num_object_copies': 14028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8546, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 194, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3867}, {'shards': '0,2', 'objects': 1342}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'11172", 'reported_seq': 11468, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:14.461459+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:14.461459+0000', 'last_peered': '2024-09-15T21:23:14.461459+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:14.461459+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29134848, 'num_objects': 7113, 'num_object_clones': 0, 'num_object_copies': 14226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3177, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9723, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 91, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1239}, {'shards': '2', 'objects': 3938}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'11359", 'reported_seq': 11646, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:14.479438+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:14.479438+0000', 'last_peered': '2024-09-15T21:23:14.479438+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:14.479438+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446144, 'num_objects': 7189, 'num_object_clones': 0, 'num_object_copies': 14378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9758, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 84, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1268}, {'shards': '2', 'objects': 3923}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11412", 'reported_seq': 11887, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.435382+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:13.435382+0000', 'last_peered': '2024-09-15T21:23:13.435382+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:13.435382+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29892608, 'num_objects': 7298, 'num_object_clones': 0, 'num_object_copies': 14596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8719, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 171, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4036}, {'shards': '0,2', 'objects': 1377}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'11252", 'reported_seq': 11690, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.416217+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:23:13.416217+0000', 'last_peered': '2024-09-15T21:23:13.416217+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:13.416217+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29327360, 'num_objects': 7160, 'num_object_clones': 0, 'num_object_copies': 14320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8837, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3933}, {'shards': '0,2', 'objects': 1337}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:16.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'11225", 'reported_seq': 11683, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.349983+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:13.349983+0000', 'last_peered': '2024-09-15T21:23:13.349983+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:13.349983+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29298688, 'num_objects': 7153, 'num_object_clones': 0, 'num_object_copies': 14306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3202, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8872, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 165, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3953}, {'shards': '0,2', 'objects': 1331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:16.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:17.507 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:17.508 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29050/230256 objects degraded (12.616%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115128,"data_bytes":471548564,"bytes_used":1972506624,"bytes_avail":287937785856,"bytes_total":289910292480,"degraded_objects":29050,"degraded_total":230256,"degraded_ratio":0.12616392189562922,"misplaced_objects":763,"misplaced_total":230256,"misplaced_ratio":0.0033137030088249602,"recovering_objects_per_sec":65,"recovering_bytes_per_sec":203386,"recovering_keys_per_sec":0,"num_objects_recovered":783,"num_bytes_recovered":2441216,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":415},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:17.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:17.563 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:18.004 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:18.007 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:18.076 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T21:23:16.430503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471548564,"num_objects":115128,"num_object_clones":0,"num_object_copies":230256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29050,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":115128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152655,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10753,"num_bytes_recovered":38125587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926276,"kb_used_data":1034372,"kb_used_omap":12,"kb_used_meta":891827,"kb_avail":281189244,"statfs":{"total":289910292480,"available":287937785856,"internally_reserved":0,"allocated":1059196928,"data_stored":1058304250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":913230858},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,6,11,2,1,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20439040,"num_objects":-4990,"num_object_clones":0,"num_object_copies":-9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":-596,"num_objects_unfound":0,"num_objects_dirty":-4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4990,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":783,"num_bytes_recovered":2441216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002836"},"pg_stats":[{"pgid":"2.f","version":"48'11290","reported_seq":11725,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.411129+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:13.411129+0000","last_peered":"2024-09-15T21:23:13.411129+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:13.411129+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11290,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4032},{"shards":"0,2","objects":1333}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11428","reported_seq":11529,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:14.432382+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:14.432382+0000","last_peered":"2024-09-15T21:23:14.432382+0000","last_clean":"2024-09-15T21:23:14.432382+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:14.432382+0000","last_undegraded":"2024-09-15T21:23:14.432382+0000","last_fullsized":"2024-09-15T21:23:14.432382+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29540352,"num_objects":7212,"num_object_clones":0,"num_object_copies":14424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10154,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11471","reported_seq":18392,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.507926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:13.507926+0000","last_peered":"2024-09-15T21:23:13.507926+0000","last_clean":"2024-09-15T21:23:13.507926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:13.507926+0000","last_undegraded":"2024-09-15T21:23:13.507926+0000","last_fullsized":"2024-09-15T21:23:13.507926+0000","mapping_epoch":45,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30302232,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10194,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11437","reported_seq":11511,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:14.422975+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:14.422975+0000","last_peered":"2024-09-15T21:23:14.422975+0000","last_clean":"2024-09-15T21:23:14.422975+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:14.422975+0000","last_undegraded":"2024-09-15T21:23:14.422975+0000","last_fullsized":"2024-09-15T21:23:14.422975+0000","mapping_epoch":36,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11456","reported_seq":11579,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.380253+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:13.380253+0000","last_peered":"2024-09-15T21:23:13.380253+0000","last_clean":"2024-09-15T21:23:13.380253+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:13.380253+0000","last_undegraded":"2024-09-15T21:23:13.380253+0000","last_fullsized":"2024-09-15T21:23:13.380253+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29765632,"num_objects":7267,"num_object_clones":0,"num_object_copies":14534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9314,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11254","reported_seq":11846,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.399808+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:13.399808+0000","last_peered":"2024-09-15T21:23:13.399808+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:13.399808+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8679,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3985},{"shards":"0,2","objects":1310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11512","reported_seq":11978,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418803+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:13.418803+0000","last_peered":"2024-09-15T21:23:13.418803+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:13.418803+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9148,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4090},{"shards":"0,2","objects":1412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11293","reported_seq":11416,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.390383+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:13.390383+0000","last_peered":"2024-09-15T21:23:13.390383+0000","last_clean":"2024-09-15T21:23:13.390383+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:13.390383+0000","last_undegraded":"2024-09-15T21:23:13.390383+0000","last_fullsized":"2024-09-15T21:23:13.390383+0000","mapping_epoch":35,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29126656,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9078,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11222","reported_seq":16260,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.500699+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:14.500699+0000","last_peered":"2024-09-15T21:23:14.500699+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:14.500699+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29298688,"num_objects":7153,"num_object_clones":0,"num_object_copies":14306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":7153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9808,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2454,"num_bytes_recovered":10018816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":763},{"shards":"2","objects":6267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11020","reported_seq":11535,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.418911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:13.418911+0000","last_peered":"2024-09-15T21:23:13.418911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:13.418911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1000","ondisk_log_start":"37'1000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1342}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11172","reported_seq":11468,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.461459+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:14.461459+0000","last_peered":"2024-09-15T21:23:14.461459+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:14.461459+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29134848,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9723,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1239},{"shards":"2","objects":3938}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11359","reported_seq":11646,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:14.479438+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:14.479438+0000","last_peered":"2024-09-15T21:23:14.479438+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:14.479438+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446144,"num_objects":7189,"num_object_clones":0,"num_object_copies":14378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9758,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1268},{"shards":"2","objects":3923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11412","reported_seq":11887,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.435382+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:13.435382+0000","last_peered":"2024-09-15T21:23:13.435382+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:13.435382+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8719,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4036},{"shards":"0,2","objects":1377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11252","reported_seq":11690,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.416217+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:13.416217+0000","last_peered":"2024-09-15T21:23:13.416217+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:13.416217+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29327360,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8837,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":1337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11225","reported_seq":11683,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.349983+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:13.349983+0000","last_peered":"2024-09-15T21:23:13.349983+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:13.349983+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29298688,"num_objects":7153,"num_object_clones":0,"num_object_copies":14306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8872,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3953},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11436","reported_seq":18525,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.483369+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:13.483369+0000","last_peered":"2024-09-15T21:23:13.483369+0000","last_clean":"2024-09-15T21:23:13.483369+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:13.483369+0000","last_undegraded":"2024-09-15T21:23:13.483369+0000","last_fullsized":"2024-09-15T21:23:13.483369+0000","mapping_epoch":47,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29418089,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9096,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471548545,"num_objects":115126,"num_object_clones":0,"num_object_copies":230252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29050,"num_objects_misplaced":763,"num_objects_unfound":0,"num_objects_dirty":115126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152653,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10752,"num_bytes_recovered":38125568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057058816,"data_stored":1056984571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953544,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591740,"kb_used_data":339300,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93780100,"statfs":{"total":96636764160,"available":96030822400,"internally_reserved":0,"allocated":347443200,"data_stored":347162838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149771,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750628,"kb_used_data":399692,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93621212,"statfs":{"total":96636764160,"available":95868121088,"internally_reserved":0,"allocated":409284608,"data_stored":409008318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":359329457},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583908,"kb_used_data":295380,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93787932,"statfs":{"total":96636764160,"available":96038842368,"internally_reserved":0,"allocated":302469120,"data_stored":302133094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":295431852},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408301568,"data_stored":408298089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301887488,"data_stored":301824273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346869760,"data_stored":346862209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:21.079 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:21.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:21.546 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:21.574 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-15T21:23:20.431315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":463147668,"num_objects":113077,"num_object_clones":0,"num_object_copies":226154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154706,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":38952979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898576,"kb_used_data":1023328,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281216944,"statfs":{"total":289910292480,"available":287966150656,"internally_reserved":0,"allocated":1047887872,"data_stored":1046995194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,4,8,3,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26112000,"num_objects":-6375,"num_object_clones":0,"num_object_copies":-12750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":-611,"num_objects_unfound":0,"num_objects_dirty":-6375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":924,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002653"},"pg_stats":[{"pgid":"2.f","version":"48'11422","reported_seq":11901,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.462772+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:18.462772+0000","last_peered":"2024-09-15T21:23:18.462772+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:18.462772+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3922},{"shards":"0,2","objects":1201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11545","reported_seq":11646,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.411461+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:19.411461+0000","last_peered":"2024-09-15T21:23:19.411461+0000","last_clean":"2024-09-15T21:23:19.411461+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:19.411461+0000","last_undegraded":"2024-09-15T21:23:19.411461+0000","last_fullsized":"2024-09-15T21:23:19.411461+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29061120,"num_objects":7095,"num_object_clones":0,"num_object_copies":14190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11592","reported_seq":18513,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.564380+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:18.564380+0000","last_peered":"2024-09-15T21:23:18.564380+0000","last_clean":"2024-09-15T21:23:18.564380+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:18.564380+0000","last_undegraded":"2024-09-15T21:23:18.564380+0000","last_fullsized":"2024-09-15T21:23:18.564380+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29806616,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10315,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11559","reported_seq":11633,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.502867+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:19.502867+0000","last_peered":"2024-09-15T21:23:19.502867+0000","last_clean":"2024-09-15T21:23:19.502867+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:19.502867+0000","last_undegraded":"2024-09-15T21:23:19.502867+0000","last_fullsized":"2024-09-15T21:23:19.502867+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11590","reported_seq":11713,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.408106+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:18.408106+0000","last_peered":"2024-09-15T21:23:18.408106+0000","last_clean":"2024-09-15T21:23:18.408106+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:18.408106+0000","last_undegraded":"2024-09-15T21:23:18.408106+0000","last_fullsized":"2024-09-15T21:23:18.408106+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11376","reported_seq":11997,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.442708+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:18.442708+0000","last_peered":"2024-09-15T21:23:18.442708+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:18.442708+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8801,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3878},{"shards":"0,2","objects":1188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11649","reported_seq":12155,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.390964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:18.390964+0000","last_peered":"2024-09-15T21:23:18.390964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:18.390964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3973},{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11409","reported_seq":11533,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.452246+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:18.452246+0000","last_peered":"2024-09-15T21:23:18.452246+0000","last_clean":"2024-09-15T21:23:18.452246+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:18.452246+0000","last_undegraded":"2024-09-15T21:23:18.452246+0000","last_fullsized":"2024-09-15T21:23:18.452246+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28651520,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11368","reported_seq":16810,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.537687+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:19.537687+0000","last_peered":"2024-09-15T21:23:19.537687+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:19.537687+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2656,"num_bytes_recovered":10846208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":561},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11148","reported_seq":11706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.440518+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:18.440518+0000","last_peered":"2024-09-15T21:23:18.440518+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:18.440518+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3761},{"shards":"0,2","objects":1214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11300","reported_seq":11608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.492469+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:19.492469+0000","last_peered":"2024-09-15T21:23:19.492469+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:19.492469+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9851,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1111},{"shards":"2","objects":3816}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11490","reported_seq":11797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.535425+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:19.535425+0000","last_peered":"2024-09-15T21:23:19.535425+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:19.535425+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1137},{"shards":"2","objects":3802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11532","reported_seq":12035,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.238742+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:18.238742+0000","last_peered":"2024-09-15T21:23:18.238742+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:18.238742+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29401088,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8839,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11380","reported_seq":11838,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.401124+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:18.401124+0000","last_peered":"2024-09-15T21:23:18.401124+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:18.401124+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11358","reported_seq":11850,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.452403+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:18.452403+0000","last_peered":"2024-09-15T21:23:18.452403+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:18.452403+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11572","reported_seq":18661,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.603474+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:18.603474+0000","last_peered":"2024-09-15T21:23:18.603474+0000","last_clean":"2024-09-15T21:23:18.603474+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:18.603474+0000","last_undegraded":"2024-09-15T21:23:18.603474+0000","last_fullsized":"2024-09-15T21:23:18.603474+0000","mapping_epoch":47,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28861033,"num_objects":7047,"num_object_clones":0,"num_object_copies":14094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":463147649,"num_objects":113075,"num_object_clones":0,"num_object_copies":226150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113075,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154704,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":38952960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1045725184,"data_stored":1045650939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160990,"ondisk_log_size":160990,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591712,"kb_used_data":335120,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93780128,"statfs":{"total":96636764160,"available":96030851072,"internally_reserved":0,"allocated":343162880,"data_stored":342882518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732104,"kb_used_data":393656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93639736,"statfs":{"total":96636764160,"available":95887089664,"internally_reserved":0,"allocated":403103744,"data_stored":402827454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574760,"kb_used_data":294552,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93797080,"statfs":{"total":96636764160,"available":96048209920,"internally_reserved":0,"allocated":301621248,"data_stored":301285222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402042880,"data_stored":402039401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301096960,"data_stored":301033745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":342585344,"data_stored":342577793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:21.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:22.047 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:22.049 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:22.081 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-15T21:23:20.431315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":463147668,"num_objects":113077,"num_object_clones":0,"num_object_copies":226154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154706,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":38952979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898576,"kb_used_data":1023328,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281216944,"statfs":{"total":289910292480,"available":287966150656,"internally_reserved":0,"allocated":1047887872,"data_stored":1046995194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,4,8,3,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26112000,"num_objects":-6375,"num_object_clones":0,"num_object_copies":-12750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-313,"num_objects_misplaced":-611,"num_objects_unfound":0,"num_objects_dirty":-6375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":924,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002653"},"pg_stats":[{"pgid":"2.f","version":"48'11422","reported_seq":11901,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.462772+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:18.462772+0000","last_peered":"2024-09-15T21:23:18.462772+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:18.462772+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3922},{"shards":"0,2","objects":1201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11545","reported_seq":11646,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.411461+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:19.411461+0000","last_peered":"2024-09-15T21:23:19.411461+0000","last_clean":"2024-09-15T21:23:19.411461+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:19.411461+0000","last_undegraded":"2024-09-15T21:23:19.411461+0000","last_fullsized":"2024-09-15T21:23:19.411461+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29061120,"num_objects":7095,"num_object_clones":0,"num_object_copies":14190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11592","reported_seq":18513,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.564380+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:18.564380+0000","last_peered":"2024-09-15T21:23:18.564380+0000","last_clean":"2024-09-15T21:23:18.564380+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:18.564380+0000","last_undegraded":"2024-09-15T21:23:18.564380+0000","last_fullsized":"2024-09-15T21:23:18.564380+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29806616,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10315,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11559","reported_seq":11633,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.502867+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:19.502867+0000","last_peered":"2024-09-15T21:23:19.502867+0000","last_clean":"2024-09-15T21:23:19.502867+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:19.502867+0000","last_undegraded":"2024-09-15T21:23:19.502867+0000","last_fullsized":"2024-09-15T21:23:19.502867+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11590","reported_seq":11713,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.408106+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:18.408106+0000","last_peered":"2024-09-15T21:23:18.408106+0000","last_clean":"2024-09-15T21:23:18.408106+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:18.408106+0000","last_undegraded":"2024-09-15T21:23:18.408106+0000","last_fullsized":"2024-09-15T21:23:18.408106+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11376","reported_seq":11997,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.442708+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:18.442708+0000","last_peered":"2024-09-15T21:23:18.442708+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:18.442708+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8801,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3878},{"shards":"0,2","objects":1188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11649","reported_seq":12155,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.390964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:18.390964+0000","last_peered":"2024-09-15T21:23:18.390964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:18.390964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3973},{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11409","reported_seq":11533,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.452246+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:18.452246+0000","last_peered":"2024-09-15T21:23:18.452246+0000","last_clean":"2024-09-15T21:23:18.452246+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:18.452246+0000","last_undegraded":"2024-09-15T21:23:18.452246+0000","last_fullsized":"2024-09-15T21:23:18.452246+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28651520,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11368","reported_seq":16810,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.537687+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:19.537687+0000","last_peered":"2024-09-15T21:23:19.537687+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:19.537687+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2656,"num_bytes_recovered":10846208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":561},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11148","reported_seq":11706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.440518+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:18.440518+0000","last_peered":"2024-09-15T21:23:18.440518+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:18.440518+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3761},{"shards":"0,2","objects":1214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11300","reported_seq":11608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.492469+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:19.492469+0000","last_peered":"2024-09-15T21:23:19.492469+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:19.492469+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9851,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1111},{"shards":"2","objects":3816}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11490","reported_seq":11797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.535425+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:19.535425+0000","last_peered":"2024-09-15T21:23:19.535425+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:19.535425+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1137},{"shards":"2","objects":3802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11532","reported_seq":12035,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.238742+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:18.238742+0000","last_peered":"2024-09-15T21:23:18.238742+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:18.238742+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29401088,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8839,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11380","reported_seq":11838,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.401124+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:18.401124+0000","last_peered":"2024-09-15T21:23:18.401124+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:18.401124+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11358","reported_seq":11850,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.452403+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:18.452403+0000","last_peered":"2024-09-15T21:23:18.452403+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:18.452403+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11572","reported_seq":18661,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.603474+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:18.603474+0000","last_peered":"2024-09-15T21:23:18.603474+0000","last_clean":"2024-09-15T21:23:18.603474+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:18.603474+0000","last_undegraded":"2024-09-15T21:23:18.603474+0000","last_fullsized":"2024-09-15T21:23:18.603474+0000","mapping_epoch":47,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28861033,"num_objects":7047,"num_object_clones":0,"num_object_copies":14094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":463147649,"num_objects":113075,"num_object_clones":0,"num_object_copies":226150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113075,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154704,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":38952960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1045725184,"data_stored":1045650939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160990,"ondisk_log_size":160990,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591712,"kb_used_data":335120,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93780128,"statfs":{"total":96636764160,"available":96030851072,"internally_reserved":0,"allocated":343162880,"data_stored":342882518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732104,"kb_used_data":393656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93639736,"statfs":{"total":96636764160,"available":95887089664,"internally_reserved":0,"allocated":403103744,"data_stored":402827454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574760,"kb_used_data":294552,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93797080,"statfs":{"total":96636764160,"available":96048209920,"internally_reserved":0,"allocated":301621248,"data_stored":301285222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402042880,"data_stored":402039401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301096960,"data_stored":301033745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":342585344,"data_stored":342577793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:22.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:22.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11422", 'reported_seq': 11901, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.462772+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:18.462772+0000', 'last_peered': '2024-09-15T21:23:18.462772+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:18.462772+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29040640, 'num_objects': 7090, 'num_object_clones': 0, 'num_object_copies': 14180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3168, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11422, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 199, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3922}, {'shards': '0,2', 'objects': 1201}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:22.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11376", 'reported_seq': 11997, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.442708+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:18.442708+0000', 'last_peered': '2024-09-15T21:23:18.442708+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:18.442708+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28692480, 'num_objects': 7005, 'num_object_clones': 0, 'num_object_copies': 14010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8801, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 244, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3878}, {'shards': '0,2', 'objects': 1188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'11649", 'reported_seq': 12155, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.390964+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:18.390964+0000', 'last_peered': '2024-09-15T21:23:18.390964+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:18.390964+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29716480, 'num_objects': 7255, 'num_object_clones': 0, 'num_object_copies': 14510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3284, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9285, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 189, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3973}, {'shards': '0,2', 'objects': 1275}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'11368", 'reported_seq': 16810, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:19.537687+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:23:19.537687+0000', 'last_peered': '2024-09-15T21:23:19.537687+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:19.537687+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28700672, 'num_objects': 7007, 'num_object_clones': 0, 'num_object_copies': 14014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 561, 'num_objects_unfound': 0, 'num_objects_dirty': 7007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9954, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2656, 'num_bytes_recovered': 10846208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 561}, {'shards': '2', 'objects': 6323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'11148", 'reported_seq': 11706, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.440518+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:18.440518+0000', 'last_peered': '2024-09-15T21:23:18.440518+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:18.440518+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1100", 'ondisk_log_start': "37'1100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28205056, 'num_objects': 6886, 'num_object_clones': 0, 'num_object_copies': 13772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3127, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6886, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8674, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 216, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3761}, {'shards': '0,2', 'objects': 1214}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'11300", 'reported_seq': 11608, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:19.492469+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:19.492469+0000', 'last_peered': '2024-09-15T21:23:19.492469+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:19.492469+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28610560, 'num_objects': 6985, 'num_object_clones': 0, 'num_object_copies': 13970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3171, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9851, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 97, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1111}, {'shards': '2', 'objects': 3816}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:22.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'11490", 'reported_seq': 11797, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:19.535425+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:19.535425+0000', 'last_peered': '2024-09-15T21:23:19.535425+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:19.535425+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28909568, 'num_objects': 7058, 'num_object_clones': 0, 'num_object_copies': 14116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3258, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9889, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1137}, {'shards': '2', 'objects': 3802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11532", 'reported_seq': 12035, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.238742+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:18.238742+0000', 'last_peered': '2024-09-15T21:23:18.238742+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:18.238742+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29401088, 'num_objects': 7178, 'num_object_clones': 0, 'num_object_copies': 14356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3249, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8839, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 185, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3930}, {'shards': '0,2', 'objects': 1257}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'11380", 'reported_seq': 11838, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.401124+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:23:18.401124+0000', 'last_peered': '2024-09-15T21:23:18.401124+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:18.401124+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28803072, 'num_objects': 7032, 'num_object_clones': 0, 'num_object_copies': 14064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3220, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8965, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 162, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3815}, {'shards': '0,2', 'objects': 1209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:22.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'11358", 'reported_seq': 11850, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.452403+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:18.452403+0000', 'last_peered': '2024-09-15T21:23:18.452403+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:18.452403+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'1300", 'ondisk_log_start': "37'1300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28753920, 'num_objects': 7020, 'num_object_clones': 0, 'num_object_copies': 14040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3185, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9005, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3837}, {'shards': '0,2', 'objects': 1198}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:22.085 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:22.658 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:22.658 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28914/226154 objects degraded (12.785%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":415,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113077,"data_bytes":463147668,"bytes_used":1944141824,"bytes_avail":287966150656,"bytes_total":289910292480,"degraded_objects":28914,"degraded_total":226154,"degraded_ratio":0.12785093343473916,"misplaced_objects":561,"misplaced_total":226154,"misplaced_ratio":0.0024806105574077843,"recovering_objects_per_sec":52,"recovering_bytes_per_sec":139225,"recovering_keys_per_sec":0,"num_objects_recovered":625,"num_bytes_recovered":1671168,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":348},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:22.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:22.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:23.213 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:23.214 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:23.266 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-15T21:23:22.432444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":463147668,"num_objects":113077,"num_object_clones":0,"num_object_copies":226154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154706,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11091,"num_bytes_recovered":38952979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160993,"ondisk_log_size":160993,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898576,"kb_used_data":1023328,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281216944,"statfs":{"total":289910292480,"available":287966150656,"internally_reserved":0,"allocated":1047887872,"data_stored":1046995194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,4,8,3,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17133568,"num_objects":-4183,"num_object_clones":0,"num_object_copies":-8366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-217,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":-4183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4183,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003361"},"pg_stats":[{"pgid":"2.f","version":"48'11422","reported_seq":11901,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.462772+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:18.462772+0000","last_peered":"2024-09-15T21:23:18.462772+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:18.462772+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3922},{"shards":"0,2","objects":1201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11545","reported_seq":11646,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.411461+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:19.411461+0000","last_peered":"2024-09-15T21:23:19.411461+0000","last_clean":"2024-09-15T21:23:19.411461+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:19.411461+0000","last_undegraded":"2024-09-15T21:23:19.411461+0000","last_fullsized":"2024-09-15T21:23:19.411461+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29061120,"num_objects":7095,"num_object_clones":0,"num_object_copies":14190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11592","reported_seq":18513,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.564380+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:18.564380+0000","last_peered":"2024-09-15T21:23:18.564380+0000","last_clean":"2024-09-15T21:23:18.564380+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:18.564380+0000","last_undegraded":"2024-09-15T21:23:18.564380+0000","last_fullsized":"2024-09-15T21:23:18.564380+0000","mapping_epoch":45,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29806616,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10315,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11559","reported_seq":11633,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:19.502867+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:19.502867+0000","last_peered":"2024-09-15T21:23:19.502867+0000","last_clean":"2024-09-15T21:23:19.502867+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:19.502867+0000","last_undegraded":"2024-09-15T21:23:19.502867+0000","last_fullsized":"2024-09-15T21:23:19.502867+0000","mapping_epoch":36,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28717056,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11559,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11590","reported_seq":11713,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.408106+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:18.408106+0000","last_peered":"2024-09-15T21:23:18.408106+0000","last_clean":"2024-09-15T21:23:18.408106+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:18.408106+0000","last_undegraded":"2024-09-15T21:23:18.408106+0000","last_fullsized":"2024-09-15T21:23:18.408106+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29216768,"num_objects":7133,"num_object_clones":0,"num_object_copies":14266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11376","reported_seq":11997,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.442708+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:18.442708+0000","last_peered":"2024-09-15T21:23:18.442708+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:18.442708+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8801,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3878},{"shards":"0,2","objects":1188}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11649","reported_seq":12155,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.390964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:18.390964+0000","last_peered":"2024-09-15T21:23:18.390964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:18.390964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3973},{"shards":"0,2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11409","reported_seq":11533,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.452246+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:18.452246+0000","last_peered":"2024-09-15T21:23:18.452246+0000","last_clean":"2024-09-15T21:23:18.452246+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:18.452246+0000","last_undegraded":"2024-09-15T21:23:18.452246+0000","last_fullsized":"2024-09-15T21:23:18.452246+0000","mapping_epoch":35,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28651520,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11368","reported_seq":16810,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.537687+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:19.537687+0000","last_peered":"2024-09-15T21:23:19.537687+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:19.537687+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2656,"num_bytes_recovered":10846208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":561},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11148","reported_seq":11706,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.440518+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:18.440518+0000","last_peered":"2024-09-15T21:23:18.440518+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:18.440518+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1100","ondisk_log_start":"37'1100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3761},{"shards":"0,2","objects":1214}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11300","reported_seq":11608,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.492469+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:19.492469+0000","last_peered":"2024-09-15T21:23:19.492469+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:19.492469+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9851,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1111},{"shards":"2","objects":3816}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11490","reported_seq":11797,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:19.535425+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:19.535425+0000","last_peered":"2024-09-15T21:23:19.535425+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:19.535425+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28909568,"num_objects":7058,"num_object_clones":0,"num_object_copies":14116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1137},{"shards":"2","objects":3802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11532","reported_seq":12035,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.238742+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:18.238742+0000","last_peered":"2024-09-15T21:23:18.238742+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:18.238742+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29401088,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8839,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3930},{"shards":"0,2","objects":1257}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11380","reported_seq":11838,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.401124+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:18.401124+0000","last_peered":"2024-09-15T21:23:18.401124+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:18.401124+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8965,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3815},{"shards":"0,2","objects":1209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11358","reported_seq":11850,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.452403+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:18.452403+0000","last_peered":"2024-09-15T21:23:18.452403+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:18.452403+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1300","ondisk_log_start":"37'1300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3837},{"shards":"0,2","objects":1198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11572","reported_seq":18661,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.603474+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:18.603474+0000","last_peered":"2024-09-15T21:23:18.603474+0000","last_clean":"2024-09-15T21:23:18.603474+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:18.603474+0000","last_undegraded":"2024-09-15T21:23:18.603474+0000","last_fullsized":"2024-09-15T21:23:18.603474+0000","mapping_epoch":47,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28861033,"num_objects":7047,"num_object_clones":0,"num_object_copies":14094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9232,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":463147649,"num_objects":113075,"num_object_clones":0,"num_object_copies":226150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28914,"num_objects_misplaced":561,"num_objects_unfound":0,"num_objects_dirty":113075,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154704,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11090,"num_bytes_recovered":38952960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1045725184,"data_stored":1045650939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160990,"ondisk_log_size":160990,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953545,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591712,"kb_used_data":335120,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93780128,"statfs":{"total":96636764160,"available":96030851072,"internally_reserved":0,"allocated":343162880,"data_stored":342882518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149772,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732104,"kb_used_data":393656,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93639736,"statfs":{"total":96636764160,"available":95887089664,"internally_reserved":0,"allocated":403103744,"data_stored":402827454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":346549937},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626491,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574760,"kb_used_data":294552,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93797080,"statfs":{"total":96636764160,"available":96048209920,"internally_reserved":0,"allocated":301621248,"data_stored":301285222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":286912172},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402042880,"data_stored":402039401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301096960,"data_stored":301033745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":342585344,"data_stored":342577793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:26.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:27.032 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:27.033 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:27.065 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-15T21:23:26.436411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":454959764,"num_objects":111078,"num_object_clones":0,"num_object_copies":222156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156705,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":40132627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900628,"kb_used_data":1012892,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281214892,"statfs":{"total":289910292480,"available":287964049408,"internally_reserved":0,"allocated":1037201408,"data_stored":1036308730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":908971018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,4,3,11,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19415040,"num_objects":-4740,"num_object_clones":0,"num_object_copies":-9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":-696,"num_objects_unfound":0,"num_objects_dirty":-4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":886,"num_bytes_recovered":2850816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006413"},"pg_stats":[{"pgid":"2.f","version":"48'11534","reported_seq":12024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.486507+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:23.486507+0000","last_peered":"2024-09-15T21:23:23.486507+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:23.486507+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":1089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11659","reported_seq":11760,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.470722+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:24.470722+0000","last_peered":"2024-09-15T21:23:24.470722+0000","last_clean":"2024-09-15T21:23:24.470722+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:24.470722+0000","last_undegraded":"2024-09-15T21:23:24.470722+0000","last_fullsized":"2024-09-15T21:23:24.470722+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11722","reported_seq":18643,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.601876+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:23.601876+0000","last_peered":"2024-09-15T21:23:23.601876+0000","last_clean":"2024-09-15T21:23:23.601876+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:23.601876+0000","last_undegraded":"2024-09-15T21:23:23.601876+0000","last_fullsized":"2024-09-15T21:23:23.601876+0000","mapping_epoch":45,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274136,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10445,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11684","reported_seq":11758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.481730+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:24.481730+0000","last_peered":"2024-09-15T21:23:24.481730+0000","last_clean":"2024-09-15T21:23:24.481730+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:24.481730+0000","last_undegraded":"2024-09-15T21:23:24.481730+0000","last_fullsized":"2024-09-15T21:23:24.481730+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11709","reported_seq":11832,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.445519+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:23.445519+0000","last_peered":"2024-09-15T21:23:23.445519+0000","last_clean":"2024-09-15T21:23:23.445519+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:23.445519+0000","last_undegraded":"2024-09-15T21:23:23.445519+0000","last_fullsized":"2024-09-15T21:23:23.445519+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11496","reported_seq":12132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.376276+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:23.376276+0000","last_peered":"2024-09-15T21:23:23.376276+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:23.376276+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3765},{"shards":"0,2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11803","reported_seq":12319,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.365745+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:23.365745+0000","last_peered":"2024-09-15T21:23:23.365745+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:23.365745+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9439,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3825},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11552","reported_seq":11676,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.448593+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:23.448593+0000","last_peered":"2024-09-15T21:23:23.448593+0000","last_clean":"2024-09-15T21:23:23.448593+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:23.448593+0000","last_undegraded":"2024-09-15T21:23:23.448593+0000","last_fullsized":"2024-09-15T21:23:23.448593+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28065792,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11495","reported_seq":17513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.535339+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:24.535339+0000","last_peered":"2024-09-15T21:23:24.535339+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:24.535339+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10081,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2944,"num_bytes_recovered":12025856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":273},{"shards":"2","objects":6484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11295","reported_seq":11866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.473949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:23.473949+0000","last_peered":"2024-09-15T21:23:23.473949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:23.473949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":1067}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11403","reported_seq":11711,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.316480+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:24.316480+0000","last_peered":"2024-09-15T21:23:24.316480+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:24.316480+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1008},{"shards":"2","objects":3713}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11610","reported_seq":11917,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.500528+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:24.500528+0000","last_peered":"2024-09-15T21:23:24.500528+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:24.500528+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10009,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1017},{"shards":"2","objects":3682}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11673","reported_seq":12200,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.474353+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:23.474353+0000","last_peered":"2024-09-15T21:23:23.474353+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:23.474353+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8980,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3801},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11491","reported_seq":11959,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.478324+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:23.478324+0000","last_peered":"2024-09-15T21:23:23.478324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:23.478324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28348416,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":1098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11466","reported_seq":11970,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.400557+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:23.400557+0000","last_peered":"2024-09-15T21:23:23.400557+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:23.400557+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9113,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11697","reported_seq":18786,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.473849+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:23.473849+0000","last_peered":"2024-09-15T21:23:23.473849+0000","last_clean":"2024-09-15T21:23:23.473849+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:23.473849+0000","last_undegraded":"2024-09-15T21:23:23.473849+0000","last_fullsized":"2024-09-15T21:23:23.473849+0000","mapping_epoch":47,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9357,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":454959745,"num_objects":111076,"num_object_clones":0,"num_object_copies":222152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156703,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":40132608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1035239424,"data_stored":1035165179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592012,"kb_used_data":331260,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93779828,"statfs":{"total":96636764160,"available":96030543872,"internally_reserved":0,"allocated":339210240,"data_stored":338929878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730128,"kb_used_data":387512,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93641712,"statfs":{"total":96636764160,"available":95889113088,"internally_reserved":0,"allocated":396812288,"data_stored":396535998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,1,2,8,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578488,"kb_used_data":294120,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93793352,"statfs":{"total":96636764160,"available":96044392448,"internally_reserved":0,"allocated":301178880,"data_stored":300842854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395821056,"data_stored":395817577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300670976,"data_stored":300607761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338747392,"data_stored":338739841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:27.067 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:27.716 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:27.717 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:27.811 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-15T21:23:26.436411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":454959764,"num_objects":111078,"num_object_clones":0,"num_object_copies":222156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156705,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":40132627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900628,"kb_used_data":1012892,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281214892,"statfs":{"total":289910292480,"available":287964049408,"internally_reserved":0,"allocated":1037201408,"data_stored":1036308730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":908971018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,4,3,11,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19415040,"num_objects":-4740,"num_object_clones":0,"num_object_copies":-9480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":-696,"num_objects_unfound":0,"num_objects_dirty":-4740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":886,"num_bytes_recovered":2850816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006413"},"pg_stats":[{"pgid":"2.f","version":"48'11534","reported_seq":12024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.486507+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:23.486507+0000","last_peered":"2024-09-15T21:23:23.486507+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:23.486507+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":1089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11659","reported_seq":11760,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.470722+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:24.470722+0000","last_peered":"2024-09-15T21:23:24.470722+0000","last_clean":"2024-09-15T21:23:24.470722+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:24.470722+0000","last_undegraded":"2024-09-15T21:23:24.470722+0000","last_fullsized":"2024-09-15T21:23:24.470722+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11722","reported_seq":18643,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.601876+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:23.601876+0000","last_peered":"2024-09-15T21:23:23.601876+0000","last_clean":"2024-09-15T21:23:23.601876+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:23.601876+0000","last_undegraded":"2024-09-15T21:23:23.601876+0000","last_fullsized":"2024-09-15T21:23:23.601876+0000","mapping_epoch":45,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274136,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10445,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11684","reported_seq":11758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.481730+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:24.481730+0000","last_peered":"2024-09-15T21:23:24.481730+0000","last_clean":"2024-09-15T21:23:24.481730+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:24.481730+0000","last_undegraded":"2024-09-15T21:23:24.481730+0000","last_fullsized":"2024-09-15T21:23:24.481730+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11709","reported_seq":11832,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.445519+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:23.445519+0000","last_peered":"2024-09-15T21:23:23.445519+0000","last_clean":"2024-09-15T21:23:23.445519+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:23.445519+0000","last_undegraded":"2024-09-15T21:23:23.445519+0000","last_fullsized":"2024-09-15T21:23:23.445519+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11496","reported_seq":12132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.376276+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:23.376276+0000","last_peered":"2024-09-15T21:23:23.376276+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:23.376276+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3765},{"shards":"0,2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11803","reported_seq":12319,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.365745+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:23.365745+0000","last_peered":"2024-09-15T21:23:23.365745+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:23.365745+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9439,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3825},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11552","reported_seq":11676,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.448593+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:23.448593+0000","last_peered":"2024-09-15T21:23:23.448593+0000","last_clean":"2024-09-15T21:23:23.448593+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:23.448593+0000","last_undegraded":"2024-09-15T21:23:23.448593+0000","last_fullsized":"2024-09-15T21:23:23.448593+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28065792,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11495","reported_seq":17513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.535339+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:24.535339+0000","last_peered":"2024-09-15T21:23:24.535339+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:24.535339+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10081,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2944,"num_bytes_recovered":12025856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":273},{"shards":"2","objects":6484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11295","reported_seq":11866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.473949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:23.473949+0000","last_peered":"2024-09-15T21:23:23.473949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:23.473949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":1067}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11403","reported_seq":11711,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.316480+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:24.316480+0000","last_peered":"2024-09-15T21:23:24.316480+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:24.316480+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1008},{"shards":"2","objects":3713}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11610","reported_seq":11917,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.500528+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:24.500528+0000","last_peered":"2024-09-15T21:23:24.500528+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:24.500528+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10009,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1017},{"shards":"2","objects":3682}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11673","reported_seq":12200,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.474353+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:23.474353+0000","last_peered":"2024-09-15T21:23:23.474353+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:23.474353+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8980,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3801},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11491","reported_seq":11959,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.478324+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:23.478324+0000","last_peered":"2024-09-15T21:23:23.478324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:23.478324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28348416,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":1098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11466","reported_seq":11970,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.400557+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:23.400557+0000","last_peered":"2024-09-15T21:23:23.400557+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:23.400557+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9113,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11697","reported_seq":18786,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.473849+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:23.473849+0000","last_peered":"2024-09-15T21:23:23.473849+0000","last_clean":"2024-09-15T21:23:23.473849+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:23.473849+0000","last_undegraded":"2024-09-15T21:23:23.473849+0000","last_fullsized":"2024-09-15T21:23:23.473849+0000","mapping_epoch":47,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9357,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":454959745,"num_objects":111076,"num_object_clones":0,"num_object_copies":222152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156703,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":40132608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1035239424,"data_stored":1035165179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592012,"kb_used_data":331260,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93779828,"statfs":{"total":96636764160,"available":96030543872,"internally_reserved":0,"allocated":339210240,"data_stored":338929878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730128,"kb_used_data":387512,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93641712,"statfs":{"total":96636764160,"available":95889113088,"internally_reserved":0,"allocated":396812288,"data_stored":396535998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,1,2,8,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578488,"kb_used_data":294120,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93793352,"statfs":{"total":96636764160,"available":96044392448,"internally_reserved":0,"allocated":301178880,"data_stored":300842854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395821056,"data_stored":395817577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300670976,"data_stored":300607761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338747392,"data_stored":338739841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:27.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:27.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11534", 'reported_seq': 12024, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.486507+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:23.486507+0000', 'last_peered': '2024-09-15T21:23:23.486507+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:23.486507+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28581888, 'num_objects': 6978, 'num_object_clones': 0, 'num_object_copies': 13956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3162, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11534, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 205, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3816}, {'shards': '0,2', 'objects': 1089}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:27.813 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11496", 'reported_seq': 12132, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.376276+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:23.376276+0000', 'last_peered': '2024-09-15T21:23:23.376276+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:23.376276+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28200960, 'num_objects': 6885, 'num_object_clones': 0, 'num_object_copies': 13770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8921, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3765}, {'shards': '0,2', 'objects': 1068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'11803", 'reported_seq': 12319, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.365745+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:23.365745+0000', 'last_peered': '2024-09-15T21:23:23.365745+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:23.365745+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29085696, 'num_objects': 7101, 'num_object_clones': 0, 'num_object_copies': 14202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3278, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9439, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 195, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3825}, {'shards': '0,2', 'objects': 1121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'11495", 'reported_seq': 17513, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:24.535339+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:23:24.535339+0000', 'last_peered': '2024-09-15T21:23:24.535339+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:24.535339+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28180480, 'num_objects': 6880, 'num_object_clones': 0, 'num_object_copies': 13760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 273, 'num_objects_unfound': 0, 'num_objects_dirty': 6880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10081, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2944, 'num_bytes_recovered': 12025856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 273}, {'shards': '2', 'objects': 6484}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'11295", 'reported_seq': 11866, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.473949+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:23.473949+0000', 'last_peered': '2024-09-15T21:23:23.473949+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:23.473949+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1200", 'ondisk_log_start': "37'1200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27602944, 'num_objects': 6739, 'num_object_clones': 0, 'num_object_copies': 13478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3121, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8821, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 222, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3620}, {'shards': '0,2', 'objects': 1067}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'11403", 'reported_seq': 11711, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:24.316480+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:24.316480+0000', 'last_peered': '2024-09-15T21:23:24.316480+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:24.316480+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28188672, 'num_objects': 6882, 'num_object_clones': 0, 'num_object_copies': 13764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3171, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9954, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 97, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1008}, {'shards': '2', 'objects': 3713}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:27.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'11610", 'reported_seq': 11917, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:24.500528+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:24.500528+0000', 'last_peered': '2024-09-15T21:23:24.500528+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:24.500528+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28418048, 'num_objects': 6938, 'num_object_clones': 0, 'num_object_copies': 13876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3258, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10009, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1017}, {'shards': '2', 'objects': 3682}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11673", 'reported_seq': 12200, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.474353+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:23.474353+0000', 'last_peered': '2024-09-15T21:23:23.474353+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:23.474353+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28823552, 'num_objects': 7037, 'num_object_clones': 0, 'num_object_copies': 14074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3237, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8980, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 197, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3801}, {'shards': '0,2', 'objects': 1116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'11491", 'reported_seq': 11959, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.478324+0000', 'last_change': '2024-09-15T21:18:43.828871+0000', 'last_active': '2024-09-15T21:23:23.478324+0000', 'last_peered': '2024-09-15T21:23:23.478324+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:23.478324+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28348416, 'num_objects': 6921, 'num_object_clones': 0, 'num_object_copies': 13842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3215, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9076, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 167, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3709}, {'shards': '0,2', 'objects': 1098}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:27.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'11466", 'reported_seq': 11970, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:23.400557+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:23.400557+0000', 'last_peered': '2024-09-15T21:23:23.400557+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:23.400557+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'1400", 'ondisk_log_start': "37'1400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28311552, 'num_objects': 6912, 'num_object_clones': 0, 'num_object_copies': 13824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9113, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 188, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3735}, {'shards': '0,2', 'objects': 1090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:27.815 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:28.619 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:28.619 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28866/222156 objects degraded (12.994%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111078,"data_bytes":454959764,"bytes_used":1946243072,"bytes_avail":287964049408,"bytes_total":289910292480,"degraded_objects":28866,"degraded_total":222156,"degraded_ratio":0.12993572084481175,"misplaced_objects":273,"misplaced_total":222156,"misplaced_ratio":0.0012288662021282342,"recovering_objects_per_sec":56,"recovering_bytes_per_sec":167164,"recovering_keys_per_sec":0,"num_objects_recovered":674,"num_bytes_recovered":2007040,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":337},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:28.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:28.694 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:29.468 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:29.470 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:29.539 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-15T21:23:28.436900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":454959764,"num_objects":111078,"num_object_clones":0,"num_object_copies":222156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156705,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":40132627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900628,"kb_used_data":1012892,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281214892,"statfs":{"total":289910292480,"available":287964049408,"internally_reserved":0,"allocated":1037201408,"data_stored":1036308730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":908971018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,4,3,11,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16588800,"num_objects":-4050,"num_object_clones":0,"num_object_copies":-8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":-490,"num_objects_unfound":0,"num_objects_dirty":-4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4050,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":2007040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006397"},"pg_stats":[{"pgid":"2.f","version":"48'11534","reported_seq":12024,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.486507+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:23.486507+0000","last_peered":"2024-09-15T21:23:23.486507+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:23.486507+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28581888,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3816},{"shards":"0,2","objects":1089}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11659","reported_seq":11760,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.470722+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:24.470722+0000","last_peered":"2024-09-15T21:23:24.470722+0000","last_clean":"2024-09-15T21:23:24.470722+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:24.470722+0000","last_undegraded":"2024-09-15T21:23:24.470722+0000","last_fullsized":"2024-09-15T21:23:24.470722+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10385,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11722","reported_seq":18643,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.601876+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:23.601876+0000","last_peered":"2024-09-15T21:23:23.601876+0000","last_clean":"2024-09-15T21:23:23.601876+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:23.601876+0000","last_undegraded":"2024-09-15T21:23:23.601876+0000","last_fullsized":"2024-09-15T21:23:23.601876+0000","mapping_epoch":45,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274136,"num_objects":7148,"num_object_clones":0,"num_object_copies":14296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10445,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11684","reported_seq":11758,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:24.481730+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:24.481730+0000","last_peered":"2024-09-15T21:23:24.481730+0000","last_clean":"2024-09-15T21:23:24.481730+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:24.481730+0000","last_undegraded":"2024-09-15T21:23:24.481730+0000","last_fullsized":"2024-09-15T21:23:24.481730+0000","mapping_epoch":36,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11709","reported_seq":11832,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.445519+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:23.445519+0000","last_peered":"2024-09-15T21:23:23.445519+0000","last_clean":"2024-09-15T21:23:23.445519+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:23.445519+0000","last_undegraded":"2024-09-15T21:23:23.445519+0000","last_fullsized":"2024-09-15T21:23:23.445519+0000","mapping_epoch":35,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9567,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11496","reported_seq":12132,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.376276+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:23.376276+0000","last_peered":"2024-09-15T21:23:23.376276+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:23.376276+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3765},{"shards":"0,2","objects":1068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11803","reported_seq":12319,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.365745+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:23.365745+0000","last_peered":"2024-09-15T21:23:23.365745+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:23.365745+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9439,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3825},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11552","reported_seq":11676,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.448593+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:23.448593+0000","last_peered":"2024-09-15T21:23:23.448593+0000","last_clean":"2024-09-15T21:23:23.448593+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:23.448593+0000","last_undegraded":"2024-09-15T21:23:23.448593+0000","last_fullsized":"2024-09-15T21:23:23.448593+0000","mapping_epoch":35,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28065792,"num_objects":6852,"num_object_clones":0,"num_object_copies":13704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11495","reported_seq":17513,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.535339+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:24.535339+0000","last_peered":"2024-09-15T21:23:24.535339+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:24.535339+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10081,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2944,"num_bytes_recovered":12025856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":273},{"shards":"2","objects":6484}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11295","reported_seq":11866,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.473949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:23.473949+0000","last_peered":"2024-09-15T21:23:23.473949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:23.473949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1200","ondisk_log_start":"37'1200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8821,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3620},{"shards":"0,2","objects":1067}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11403","reported_seq":11711,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.316480+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:24.316480+0000","last_peered":"2024-09-15T21:23:24.316480+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:24.316480+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9954,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1008},{"shards":"2","objects":3713}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11610","reported_seq":11917,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:24.500528+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:24.500528+0000","last_peered":"2024-09-15T21:23:24.500528+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:24.500528+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10009,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1017},{"shards":"2","objects":3682}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11673","reported_seq":12200,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.474353+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:23.474353+0000","last_peered":"2024-09-15T21:23:23.474353+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:23.474353+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8980,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3801},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11491","reported_seq":11959,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.478324+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:23.478324+0000","last_peered":"2024-09-15T21:23:23.478324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:23.478324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28348416,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9076,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":1098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11466","reported_seq":11970,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.400557+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:23.400557+0000","last_peered":"2024-09-15T21:23:23.400557+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:23.400557+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28311552,"num_objects":6912,"num_object_clones":0,"num_object_copies":13824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9113,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":1090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11697","reported_seq":18786,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.473849+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:23.473849+0000","last_peered":"2024-09-15T21:23:23.473849+0000","last_clean":"2024-09-15T21:23:23.473849+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:23.473849+0000","last_undegraded":"2024-09-15T21:23:23.473849+0000","last_fullsized":"2024-09-15T21:23:23.473849+0000","mapping_epoch":47,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9357,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":454959745,"num_objects":111076,"num_object_clones":0,"num_object_copies":222152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28866,"num_objects_misplaced":273,"num_objects_unfound":0,"num_objects_dirty":111076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156703,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":40132608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1035239424,"data_stored":1035165179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592012,"kb_used_data":331260,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93779828,"statfs":{"total":96636764160,"available":96030543872,"internally_reserved":0,"allocated":339210240,"data_stored":338929878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149773,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730128,"kb_used_data":387512,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93641712,"statfs":{"total":96636764160,"available":95889113088,"internally_reserved":0,"allocated":396812288,"data_stored":396535998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":350809777},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[14,3,1,2,8,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578488,"kb_used_data":294120,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93793352,"statfs":{"total":96636764160,"available":96044392448,"internally_reserved":0,"allocated":301178880,"data_stored":300842854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395821056,"data_stored":395817577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300670976,"data_stored":300607761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338747392,"data_stored":338739841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:32.541 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:33.226 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:33.226 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:33.251 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-15T21:23:32.438043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446063252,"num_objects":108906,"num_object_clones":0,"num_object_copies":217812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28765,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":108906,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158877,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11742,"num_bytes_recovered":41009171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1876560,"kb_used_data":1001304,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281238960,"statfs":{"total":289910292480,"available":287988695040,"internally_reserved":0,"allocated":1025335296,"data_stored":1024442618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":896191498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,2,9,3,1,0,0,1,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17084416,"num_objects":-4171,"num_object_clones":0,"num_object_copies":-8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":-502,"num_objects_unfound":0,"num_objects_dirty":-4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4171,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":651,"num_bytes_recovered":2056192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006728"},"pg_stats":[{"pgid":"2.f","version":"48'11673","reported_seq":12212,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.427506+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:28.427506+0000","last_peered":"2024-09-15T21:23:28.427506+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:28.427506+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11673,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3701},{"shards":"0,2","objects":950}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11782","reported_seq":11882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:29.510507+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:29.510507+0000","last_peered":"2024-09-15T21:23:29.510507+0000","last_clean":"2024-09-15T21:23:29.510507+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:29.510507+0000","last_undegraded":"2024-09-15T21:23:29.510507+0000","last_fullsized":"2024-09-15T21:23:29.510507+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11874","reported_seq":18795,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.754113+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:28.754113+0000","last_peered":"2024-09-15T21:23:28.754113+0000","last_clean":"2024-09-15T21:23:28.754113+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:28.754113+0000","last_undegraded":"2024-09-15T21:23:28.754113+0000","last_fullsized":"2024-09-15T21:23:28.754113+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28651544,"num_objects":6996,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10597,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11816","reported_seq":11890,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:29.555756+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:29.555756+0000","last_peered":"2024-09-15T21:23:29.555756+0000","last_clean":"2024-09-15T21:23:29.555756+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:29.555756+0000","last_undegraded":"2024-09-15T21:23:29.555756+0000","last_fullsized":"2024-09-15T21:23:29.555756+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27664384,"num_objects":6754,"num_object_clones":0,"num_object_copies":13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11816,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11840","reported_seq":11963,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.450791+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:28.450791+0000","last_peered":"2024-09-15T21:23:28.450791+0000","last_clean":"2024-09-15T21:23:28.450791+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:28.450791+0000","last_undegraded":"2024-09-15T21:23:28.450791+0000","last_fullsized":"2024-09-15T21:23:28.450791+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28192768,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9698,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11619","reported_seq":12267,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.371711+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:28.371711+0000","last_peered":"2024-09-15T21:23:28.371711+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:28.371711+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27697152,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9044,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3648},{"shards":"0,2","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'11933","reported_seq":12483,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.352449+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:28.352449+0000","last_peered":"2024-09-15T21:23:28.352449+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:28.352449+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28553216,"num_objects":6971,"num_object_clones":0,"num_object_copies":13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9569,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3711},{"shards":"0,2","objects":991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11692","reported_seq":11816,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.453065+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:28.453065+0000","last_peered":"2024-09-15T21:23:28.453065+0000","last_clean":"2024-09-15T21:23:28.453065+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:28.453065+0000","last_undegraded":"2024-09-15T21:23:28.453065+0000","last_fullsized":"2024-09-15T21:23:28.453065+0000","mapping_epoch":35,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27492352,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9477,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11631","reported_seq":18077,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.553863+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:29.553863+0000","last_peered":"2024-09-15T21:23:29.553863+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:29.553863+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10217,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":12902400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":59},{"shards":"2","objects":6562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11437","reported_seq":12022,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.421509+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:28.421509+0000","last_peered":"2024-09-15T21:23:28.421509+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:28.421509+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1400","ondisk_log_start":"37'1400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27021312,"num_objects":6597,"num_object_clones":0,"num_object_copies":13194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11525","reported_seq":11861,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.350194+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:29.350194+0000","last_peered":"2024-09-15T21:23:29.350194+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:29.350194+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10076,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":886},{"shards":"2","objects":3605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11732","reported_seq":12045,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.517257+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:29.517257+0000","last_peered":"2024-09-15T21:23:29.517257+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:29.517257+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27918336,"num_objects":6816,"num_object_clones":0,"num_object_copies":13632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10131,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":3563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11828","reported_seq":12389,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.457668+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:28.457668+0000","last_peered":"2024-09-15T21:23:28.457668+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:28.457668+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28188672,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9135,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3663},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11637","reported_seq":12123,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.386151+0000","last_change":"2024-09-15T21:18:43.828871+0000","last_active":"2024-09-15T21:23:28.386151+0000","last_peered":"2024-09-15T21:23:28.386151+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:28.386151+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9222,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3572},{"shards":"0,2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11616","reported_seq":12130,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.479846+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:28.479846+0000","last_peered":"2024-09-15T21:23:28.479846+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:28.479846+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27697152,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9263,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3590},{"shards":"0,2","objects":940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11826","reported_seq":18915,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.647022+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:28.647022+0000","last_peered":"2024-09-15T21:23:28.647022+0000","last_clean":"2024-09-15T21:23:28.647022+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:28.647022+0000","last_undegraded":"2024-09-15T21:23:28.647022+0000","last_fullsized":"2024-09-15T21:23:28.647022+0000","mapping_epoch":47,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820649,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446063233,"num_objects":108904,"num_object_clones":0,"num_object_copies":217808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28765,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":108904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158875,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11741,"num_bytes_recovered":41009152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023275008,"data_stored":1023200763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160661,"ondisk_log_size":160661,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575412,"kb_used_data":327132,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93796428,"statfs":{"total":96636764160,"available":96047542272,"internally_reserved":0,"allocated":334983168,"data_stored":334702806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149774,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715148,"kb_used_data":380860,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93656692,"statfs":{"total":96636764160,"available":95904452608,"internally_reserved":0,"allocated":390000640,"data_stored":389724350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":342290097},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,7,1,1,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586000,"kb_used_data":293312,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93785840,"statfs":{"total":96636764160,"available":96036700160,"internally_reserved":0,"allocated":300351488,"data_stored":300015462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389099520,"data_stored":389096041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299790336,"data_stored":299727121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334385152,"data_stored":334377601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:33.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:33.967 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:33.968 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:34.014 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-15T21:23:33.632614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441049748,"num_objects":107682,"num_object_clones":0,"num_object_copies":215364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27853,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":107682,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160101,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11845,"num_bytes_recovered":41017363,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892192,"kb_used_data":994724,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":281223328,"statfs":{"total":289910292480,"available":287972687872,"internally_reserved":0,"allocated":1018597376,"data_stored":1017704698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":918932490},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,8,5,2,0,4,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22097920,"num_objects":-5395,"num_object_clones":0,"num_object_copies":-10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":307,"num_objects_unfound":0,"num_objects_dirty":-5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":754,"num_bytes_recovered":2064384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.200169"},"pg_stats":[{"pgid":"2.f","version":"48'11804","reported_seq":12379,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.477629+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:33.477629+0000","last_peered":"2024-09-15T21:23:33.477629+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:33.477629+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11804,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"48'11782","reported_seq":11882,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:29.510507+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:29.510507+0000","last_peered":"2024-09-15T21:23:29.510507+0000","last_clean":"2024-09-15T21:23:29.510507+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:29.510507+0000","last_undegraded":"2024-09-15T21:23:29.510507+0000","last_fullsized":"2024-09-15T21:23:29.510507+0000","mapping_epoch":36,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"48'11874","reported_seq":18795,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.754113+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:28.754113+0000","last_peered":"2024-09-15T21:23:28.754113+0000","last_clean":"2024-09-15T21:23:28.754113+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:28.754113+0000","last_undegraded":"2024-09-15T21:23:28.754113+0000","last_fullsized":"2024-09-15T21:23:28.754113+0000","mapping_epoch":45,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28651544,"num_objects":6996,"num_object_clones":0,"num_object_copies":13992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10597,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"48'11816","reported_seq":11890,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:29.555756+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:29.555756+0000","last_peered":"2024-09-15T21:23:29.555756+0000","last_clean":"2024-09-15T21:23:29.555756+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:29.555756+0000","last_undegraded":"2024-09-15T21:23:29.555756+0000","last_fullsized":"2024-09-15T21:23:29.555756+0000","mapping_epoch":36,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27664384,"num_objects":6754,"num_object_clones":0,"num_object_copies":13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11816,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11965","reported_seq":12088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.463460+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:33.463460+0000","last_peered":"2024-09-15T21:23:33.463460+0000","last_clean":"2024-09-15T21:23:33.463460+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:33.463460+0000","last_undegraded":"2024-09-15T21:23:33.463460+0000","last_fullsized":"2024-09-15T21:23:33.463460+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27680768,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11761","reported_seq":12435,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.518469+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:33.518469+0000","last_peered":"2024-09-15T21:23:33.518469+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:33.518469+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9186,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3519},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'12058","reported_seq":12636,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.460460+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:33.460460+0000","last_peered":"2024-09-15T21:23:33.460460+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:33.460460+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11831","reported_seq":11955,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.481597+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:33.481597+0000","last_peered":"2024-09-15T21:23:33.481597+0000","last_clean":"2024-09-15T21:23:33.481597+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:33.481597+0000","last_undegraded":"2024-09-15T21:23:33.481597+0000","last_fullsized":"2024-09-15T21:23:33.481597+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9616,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.548123+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:22:00.548123+0000","last_peered":"2024-09-15T21:22:00.548123+0000","last_clean":"2024-09-15T21:22:00.548123+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:22:00.548123+0000","last_undegraded":"2024-09-15T21:22:00.548123+0000","last_fullsized":"2024-09-15T21:22:00.548123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11631","reported_seq":18077,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.553863+0000","last_change":"2024-09-15T21:21:58.787788+0000","last_active":"2024-09-15T21:23:29.553863+0000","last_peered":"2024-09-15T21:23:29.553863+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:29.553863+0000","last_undegraded":"2024-09-15T21:18:43.442399+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10217,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":12902400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":59},{"shards":"2","objects":6562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.549078+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:22:00.549078+0000","last_peered":"2024-09-15T21:22:00.549078+0000","last_clean":"2024-09-15T21:22:00.549078+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:22:00.549078+0000","last_undegraded":"2024-09-15T21:22:00.549078+0000","last_fullsized":"2024-09-15T21:22:00.549078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11577","reported_seq":12196,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.493175+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:33.493175+0000","last_peered":"2024-09-15T21:23:33.493175+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:33.493175+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9103,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3362},{"shards":"0,2","objects":785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"48'11525","reported_seq":11861,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.350194+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:29.350194+0000","last_peered":"2024-09-15T21:23:29.350194+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:29.350194+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10076,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":886},{"shards":"2","objects":3605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551335+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:22:00.551335+0000","last_peered":"2024-09-15T21:22:00.551335+0000","last_clean":"2024-09-15T21:22:00.551335+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:22:00.551335+0000","last_undegraded":"2024-09-15T21:22:00.551335+0000","last_fullsized":"2024-09-15T21:22:00.551335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"48'11732","reported_seq":12045,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:29.517257+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:29.517257+0000","last_peered":"2024-09-15T21:23:29.517257+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:29.517257+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27918336,"num_objects":6816,"num_object_clones":0,"num_object_copies":13632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10131,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":3563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11966","reported_seq":12555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.480938+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:33.480938+0000","last_peered":"2024-09-15T21:23:33.480938+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:33.480938+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9273,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551998+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:22:00.551998+0000","last_peered":"2024-09-15T21:22:00.551998+0000","last_clean":"2024-09-15T21:22:00.551998+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:22:00.551998+0000","last_undegraded":"2024-09-15T21:22:00.551998+0000","last_fullsized":"2024-09-15T21:22:00.551998+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11779","reported_seq":12282,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.524327+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:33.524327+0000","last_peered":"2024-09-15T21:23:33.524327+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:33.524327+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9364,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.551911+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:22:00.551911+0000","last_peered":"2024-09-15T21:22:00.551911+0000","last_clean":"2024-09-15T21:22:00.551911+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:22:00.551911+0000","last_undegraded":"2024-09-15T21:22:00.551911+0000","last_fullsized":"2024-09-15T21:22:00.551911+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11758","reported_seq":12310,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.410667+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:33.410667+0000","last_peered":"2024-09-15T21:23:33.410667+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:33.410667+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9405,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552662+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:22:00.552662+0000","last_peered":"2024-09-15T21:22:00.552662+0000","last_clean":"2024-09-15T21:22:00.552662+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:22:00.552662+0000","last_undegraded":"2024-09-15T21:22:00.552662+0000","last_fullsized":"2024-09-15T21:22:00.552662+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.552684+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:22:00.552684+0000","last_peered":"2024-09-15T21:22:00.552684+0000","last_clean":"2024-09-15T21:22:00.552684+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:22:00.552684+0000","last_undegraded":"2024-09-15T21:22:00.552684+0000","last_fullsized":"2024-09-15T21:22:00.552684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"48'11826","reported_seq":18915,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.647022+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:28.647022+0000","last_peered":"2024-09-15T21:23:28.647022+0000","last_clean":"2024-09-15T21:23:28.647022+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:28.647022+0000","last_undegraded":"2024-09-15T21:23:28.647022+0000","last_fullsized":"2024-09-15T21:23:28.647022+0000","mapping_epoch":47,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820649,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441049729,"num_objects":107680,"num_object_clones":0,"num_object_copies":215360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27853,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":107680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160099,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11844,"num_bytes_recovered":41017344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1016516608,"data_stored":1016442363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575412,"kb_used_data":327132,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93796428,"statfs":{"total":96636764160,"available":96047542272,"internally_reserved":0,"allocated":334983168,"data_stored":334702806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730780,"kb_used_data":374280,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93641060,"statfs":{"total":96636764160,"available":95888445440,"internally_reserved":0,"allocated":383262720,"data_stored":382986430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586000,"kb_used_data":293312,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93785840,"statfs":{"total":96636764160,"available":96036700160,"internally_reserved":0,"allocated":300351488,"data_stored":300015462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":299691692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382341120,"data_stored":382337641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299790336,"data_stored":299727121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334385152,"data_stored":334377601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:34.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:34.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11804", 'reported_seq': 12379, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.477629+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:33.477629+0000', 'last_peered': '2024-09-15T21:23:33.477629+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:33.477629+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27475968, 'num_objects': 6708, 'num_object_clones': 0, 'num_object_copies': 13416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11804, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 247, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3588}, {'shards': '0,2', 'objects': 819}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:34.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11761", 'reported_seq': 12435, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.518469+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:33.518469+0000', 'last_peered': '2024-09-15T21:23:33.518469+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:33.518469+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27115520, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9186, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 270, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3519}, {'shards': '0,2', 'objects': 803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'12058", 'reported_seq': 12636, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.460460+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:33.460460+0000', 'last_peered': '2024-09-15T21:23:33.460460+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:33.460460+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28041216, 'num_objects': 6846, 'num_object_clones': 0, 'num_object_copies': 13692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9694, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 225, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3600}, {'shards': '0,2', 'objects': 866}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'11631", 'reported_seq': 18077, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:29.553863+0000', 'last_change': '2024-09-15T21:21:58.787788+0000', 'last_active': '2024-09-15T21:23:29.553863+0000', 'last_peered': '2024-09-15T21:23:29.553863+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:29.553863+0000', 'last_undegraded': '2024-09-15T21:18:43.442399+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27623424, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 123, 'num_objects_misplaced': 59, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10217, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3158, 'num_bytes_recovered': 12902400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 59}, {'shards': '2', 'objects': 6562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'11577", 'reported_seq': 12196, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.493175+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:33.493175+0000', 'last_peered': '2024-09-15T21:23:33.493175+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:33.493175+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26447872, 'num_objects': 6457, 'num_object_clones': 0, 'num_object_copies': 12914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9103, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 246, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3362}, {'shards': '0,2', 'objects': 785}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "48'11525", 'reported_seq': 11861, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:29.350194+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:29.350194+0000', 'last_peered': '2024-09-15T21:23:29.350194+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:29.350194+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27688960, 'num_objects': 6760, 'num_object_clones': 0, 'num_object_copies': 13520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6760, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10076, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 111, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 886}, {'shards': '2', 'objects': 3605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "48'11732", 'reported_seq': 12045, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:29.517257+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:29.517257+0000', 'last_peered': '2024-09-15T21:23:29.517257+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:29.517257+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27918336, 'num_objects': 6816, 'num_object_clones': 0, 'num_object_copies': 13632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3255, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10131, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 97, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 895}, {'shards': '2', 'objects': 3563}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:34.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:34.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11966", 'reported_seq': 12555, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.480938+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:33.480938+0000', 'last_peered': '2024-09-15T21:23:33.480938+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:33.480938+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27623424, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9273, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 228, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3539}, {'shards': '0,2', 'objects': 823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:34.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'11779", 'reported_seq': 12282, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.524327+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:33.524327+0000', 'last_peered': '2024-09-15T21:23:33.524327+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:33.524327+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27168768, 'num_objects': 6633, 'num_object_clones': 0, 'num_object_copies': 13266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2389, 'num_objects_misplaced': 809, 'num_objects_unfound': 0, 'num_objects_dirty': 6633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9364, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 184, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3435}, {'shards': '0,2', 'objects': 809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:34.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'11758", 'reported_seq': 12310, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.410667+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:33.410667+0000', 'last_peered': '2024-09-15T21:23:33.410667+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:33.410667+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27115520, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9405, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 212, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3467}, {'shards': '0,2', 'objects': 798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:34.019 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:34.553 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:34.553 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28765/217812 objects degraded (13.206%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":427,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108906,"data_bytes":446063252,"bytes_used":1921597440,"bytes_avail":287988695040,"bytes_total":289910292480,"degraded_objects":28765,"degraded_total":217812,"degraded_ratio":0.13206343084862174,"misplaced_objects":59,"misplaced_total":217812,"misplaced_ratio":0.00027087580114961525,"recovering_objects_per_sec":54,"recovering_bytes_per_sec":171253,"recovering_keys_per_sec":0,"num_objects_recovered":651,"num_bytes_recovered":2056192,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":347},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:34.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:34.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:35.135 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:35.135 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:35.176 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-15T21:23:34.668691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437088916,"num_objects":106715,"num_object_clones":0,"num_object_copies":213430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161068,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12042,"num_bytes_recovered":41259027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882288,"kb_used_data":988984,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":281233232,"statfs":{"total":289910292480,"available":287982829568,"internally_reserved":0,"allocated":1012719616,"data_stored":1011821701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":914672650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,5,10,5,3,0,4,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17870848,"num_objects":-4363,"num_object_clones":0,"num_object_copies":-8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1151,"num_objects_misplaced":536,"num_objects_unfound":0,"num_objects_dirty":-4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.232279"},"pg_stats":[{"pgid":"2.f","version":"48'11804","reported_seq":12379,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.477629+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:33.477629+0000","last_peered":"2024-09-15T21:23:33.477629+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:33.477629+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11804,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11922","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.531235+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:34.531235+0000","last_peered":"2024-09-15T21:23:34.531235+0000","last_clean":"2024-09-15T21:23:34.531235+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:34.531235+0000","last_undegraded":"2024-09-15T21:23:34.531235+0000","last_fullsized":"2024-09-15T21:23:34.531235+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12029","reported_seq":18949,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.734669+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:33.734669+0000","last_peered":"2024-09-15T21:23:33.734669+0000","last_clean":"2024-09-15T21:23:33.734669+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:33.734669+0000","last_undegraded":"2024-09-15T21:23:33.734669+0000","last_fullsized":"2024-09-15T21:23:33.734669+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016664,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10752,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'11949","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.527424+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:34.527424+0000","last_peered":"2024-09-15T21:23:34.527424+0000","last_clean":"2024-09-15T21:23:34.527424+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:34.527424+0000","last_undegraded":"2024-09-15T21:23:34.527424+0000","last_fullsized":"2024-09-15T21:23:34.527424+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11949,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11965","reported_seq":12088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.463460+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:33.463460+0000","last_peered":"2024-09-15T21:23:33.463460+0000","last_clean":"2024-09-15T21:23:33.463460+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:33.463460+0000","last_undegraded":"2024-09-15T21:23:33.463460+0000","last_fullsized":"2024-09-15T21:23:33.463460+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27680768,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11761","reported_seq":12435,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.518469+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:33.518469+0000","last_peered":"2024-09-15T21:23:33.518469+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:33.518469+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9186,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3519},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'12058","reported_seq":12636,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.460460+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:33.460460+0000","last_peered":"2024-09-15T21:23:33.460460+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:33.460460+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11831","reported_seq":11955,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.481597+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:33.481597+0000","last_peered":"2024-09-15T21:23:33.481597+0000","last_clean":"2024-09-15T21:23:33.481597+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:33.481597+0000","last_undegraded":"2024-09-15T21:23:33.481597+0000","last_fullsized":"2024-09-15T21:23:33.481597+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9616,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.645356+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:33.645356+0000","last_peered":"2024-09-15T21:23:33.645356+0000","last_clean":"2024-09-15T21:23:33.645356+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:33.645356+0000","last_undegraded":"2024-09-15T21:23:33.645356+0000","last_fullsized":"2024-09-15T21:23:33.645356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11756","reported_seq":18570,"reported_epoch":49,"state":"peering","last_fresh":"2024-09-15T21:23:33.614574+0000","last_change":"2024-09-15T21:23:33.778706+0000","last_active":"2024-09-15T21:23:33.778706+0000","last_peered":"2024-09-15T21:23:33.614574+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:33.614574+0000","last_undegraded":"2024-09-15T21:23:33.614574+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":49,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10342,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.639707+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:33.639707+0000","last_peered":"2024-09-15T21:23:33.639707+0000","last_clean":"2024-09-15T21:23:33.639707+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:33.639707+0000","last_undegraded":"2024-09-15T21:23:33.639707+0000","last_fullsized":"2024-09-15T21:23:33.639707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11577","reported_seq":12196,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.493175+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:33.493175+0000","last_peered":"2024-09-15T21:23:33.493175+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:33.493175+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9103,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3362},{"shards":"0,2","objects":785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11662","reported_seq":12006,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.519204+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:34.519204+0000","last_peered":"2024-09-15T21:23:34.519204+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:34.519204+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":86,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641436+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:33.641436+0000","last_peered":"2024-09-15T21:23:33.641436+0000","last_clean":"2024-09-15T21:23:33.641436+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:33.641436+0000","last_undegraded":"2024-09-15T21:23:33.641436+0000","last_fullsized":"2024-09-15T21:23:33.641436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11878","reported_seq":12217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.443838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:34.443838+0000","last_peered":"2024-09-15T21:23:34.443838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:34.443838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11966","reported_seq":12555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.480938+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:33.480938+0000","last_peered":"2024-09-15T21:23:33.480938+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:33.480938+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9273,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.642773+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:33.642773+0000","last_peered":"2024-09-15T21:23:33.642773+0000","last_clean":"2024-09-15T21:23:33.642773+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:33.642773+0000","last_undegraded":"2024-09-15T21:23:33.642773+0000","last_fullsized":"2024-09-15T21:23:33.642773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11779","reported_seq":12282,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.524327+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:33.524327+0000","last_peered":"2024-09-15T21:23:33.524327+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:33.524327+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9364,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.640890+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:33.640890+0000","last_peered":"2024-09-15T21:23:33.640890+0000","last_clean":"2024-09-15T21:23:33.640890+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:33.640890+0000","last_undegraded":"2024-09-15T21:23:33.640890+0000","last_fullsized":"2024-09-15T21:23:33.640890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11758","reported_seq":12310,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.410667+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:33.410667+0000","last_peered":"2024-09-15T21:23:33.410667+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:33.410667+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9405,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641594+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:33.641594+0000","last_peered":"2024-09-15T21:23:33.641594+0000","last_clean":"2024-09-15T21:23:33.641594+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:33.641594+0000","last_undegraded":"2024-09-15T21:23:33.641594+0000","last_fullsized":"2024-09-15T21:23:33.641594+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.646790+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:33.646790+0000","last_peered":"2024-09-15T21:23:33.646790+0000","last_clean":"2024-09-15T21:23:33.646790+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:33.646790+0000","last_undegraded":"2024-09-15T21:23:33.646790+0000","last_fullsized":"2024-09-15T21:23:33.646790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'11957","reported_seq":19048,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.703107+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:33.703107+0000","last_peered":"2024-09-15T21:23:33.703107+0000","last_clean":"2024-09-15T21:23:33.703107+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:33.703107+0000","last_undegraded":"2024-09-15T21:23:33.703107+0000","last_fullsized":"2024-09-15T21:23:33.703107+0000","mapping_epoch":47,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27284073,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437088897,"num_objects":106713,"num_object_clones":0,"num_object_copies":213426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161066,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12041,"num_bytes_recovered":41259008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010733056,"data_stored":1010669248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575088,"kb_used_data":322652,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93796752,"statfs":{"total":96636764160,"available":96047874048,"internally_reserved":0,"allocated":330395648,"data_stored":330110049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,1,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730780,"kb_used_data":374280,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93641060,"statfs":{"total":96636764160,"available":95888445440,"internally_reserved":0,"allocated":383262720,"data_stored":382986430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576420,"kb_used_data":292052,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93795420,"statfs":{"total":96636764160,"available":96046510080,"internally_reserved":0,"allocated":299061248,"data_stored":298725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382341120,"data_stored":382337641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298426368,"data_stored":298373590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329965568,"data_stored":329958017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:38.178 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:38.923 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:38.924 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:38.990 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T21:23:38.439239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437088916,"num_objects":106715,"num_object_clones":0,"num_object_copies":213430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161068,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12042,"num_bytes_recovered":41259027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882288,"kb_used_data":988984,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":281233232,"statfs":{"total":289910292480,"available":287982829568,"internally_reserved":0,"allocated":1012719616,"data_stored":1011821701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":914672650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,5,10,5,3,0,4,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8974336,"num_objects":-2191,"num_object_clones":0,"num_object_copies":-4382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1050,"num_objects_misplaced":750,"num_objects_unfound":0,"num_objects_dirty":-2191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2191,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":249856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001724"},"pg_stats":[{"pgid":"2.f","version":"48'11804","reported_seq":12379,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.477629+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:33.477629+0000","last_peered":"2024-09-15T21:23:33.477629+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:33.477629+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11804,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11922","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.531235+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:34.531235+0000","last_peered":"2024-09-15T21:23:34.531235+0000","last_clean":"2024-09-15T21:23:34.531235+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:34.531235+0000","last_undegraded":"2024-09-15T21:23:34.531235+0000","last_fullsized":"2024-09-15T21:23:34.531235+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12029","reported_seq":18949,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.734669+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:33.734669+0000","last_peered":"2024-09-15T21:23:33.734669+0000","last_clean":"2024-09-15T21:23:33.734669+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:33.734669+0000","last_undegraded":"2024-09-15T21:23:33.734669+0000","last_fullsized":"2024-09-15T21:23:33.734669+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016664,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10752,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'11949","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.527424+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:34.527424+0000","last_peered":"2024-09-15T21:23:34.527424+0000","last_clean":"2024-09-15T21:23:34.527424+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:34.527424+0000","last_undegraded":"2024-09-15T21:23:34.527424+0000","last_fullsized":"2024-09-15T21:23:34.527424+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11949,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11965","reported_seq":12088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.463460+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:33.463460+0000","last_peered":"2024-09-15T21:23:33.463460+0000","last_clean":"2024-09-15T21:23:33.463460+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:33.463460+0000","last_undegraded":"2024-09-15T21:23:33.463460+0000","last_fullsized":"2024-09-15T21:23:33.463460+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27680768,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11761","reported_seq":12435,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.518469+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:33.518469+0000","last_peered":"2024-09-15T21:23:33.518469+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:33.518469+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9186,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3519},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'12058","reported_seq":12636,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.460460+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:33.460460+0000","last_peered":"2024-09-15T21:23:33.460460+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:33.460460+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11831","reported_seq":11955,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.481597+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:33.481597+0000","last_peered":"2024-09-15T21:23:33.481597+0000","last_clean":"2024-09-15T21:23:33.481597+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:33.481597+0000","last_undegraded":"2024-09-15T21:23:33.481597+0000","last_fullsized":"2024-09-15T21:23:33.481597+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9616,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.645356+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:33.645356+0000","last_peered":"2024-09-15T21:23:33.645356+0000","last_clean":"2024-09-15T21:23:33.645356+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:33.645356+0000","last_undegraded":"2024-09-15T21:23:33.645356+0000","last_fullsized":"2024-09-15T21:23:33.645356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11756","reported_seq":18570,"reported_epoch":49,"state":"peering","last_fresh":"2024-09-15T21:23:33.614574+0000","last_change":"2024-09-15T21:23:33.778706+0000","last_active":"2024-09-15T21:23:33.778706+0000","last_peered":"2024-09-15T21:23:33.614574+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:33.614574+0000","last_undegraded":"2024-09-15T21:23:33.614574+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":49,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10342,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.639707+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:33.639707+0000","last_peered":"2024-09-15T21:23:33.639707+0000","last_clean":"2024-09-15T21:23:33.639707+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:33.639707+0000","last_undegraded":"2024-09-15T21:23:33.639707+0000","last_fullsized":"2024-09-15T21:23:33.639707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11577","reported_seq":12196,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.493175+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:33.493175+0000","last_peered":"2024-09-15T21:23:33.493175+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:33.493175+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9103,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3362},{"shards":"0,2","objects":785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11662","reported_seq":12006,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.519204+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:34.519204+0000","last_peered":"2024-09-15T21:23:34.519204+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:34.519204+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":86,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641436+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:33.641436+0000","last_peered":"2024-09-15T21:23:33.641436+0000","last_clean":"2024-09-15T21:23:33.641436+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:33.641436+0000","last_undegraded":"2024-09-15T21:23:33.641436+0000","last_fullsized":"2024-09-15T21:23:33.641436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11878","reported_seq":12217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.443838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:34.443838+0000","last_peered":"2024-09-15T21:23:34.443838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:34.443838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11966","reported_seq":12555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.480938+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:33.480938+0000","last_peered":"2024-09-15T21:23:33.480938+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:33.480938+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9273,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.642773+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:33.642773+0000","last_peered":"2024-09-15T21:23:33.642773+0000","last_clean":"2024-09-15T21:23:33.642773+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:33.642773+0000","last_undegraded":"2024-09-15T21:23:33.642773+0000","last_fullsized":"2024-09-15T21:23:33.642773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11779","reported_seq":12282,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.524327+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:33.524327+0000","last_peered":"2024-09-15T21:23:33.524327+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:33.524327+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9364,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.640890+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:33.640890+0000","last_peered":"2024-09-15T21:23:33.640890+0000","last_clean":"2024-09-15T21:23:33.640890+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:33.640890+0000","last_undegraded":"2024-09-15T21:23:33.640890+0000","last_fullsized":"2024-09-15T21:23:33.640890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11758","reported_seq":12310,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.410667+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:33.410667+0000","last_peered":"2024-09-15T21:23:33.410667+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:33.410667+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9405,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641594+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:33.641594+0000","last_peered":"2024-09-15T21:23:33.641594+0000","last_clean":"2024-09-15T21:23:33.641594+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:33.641594+0000","last_undegraded":"2024-09-15T21:23:33.641594+0000","last_fullsized":"2024-09-15T21:23:33.641594+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.646790+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:33.646790+0000","last_peered":"2024-09-15T21:23:33.646790+0000","last_clean":"2024-09-15T21:23:33.646790+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:33.646790+0000","last_undegraded":"2024-09-15T21:23:33.646790+0000","last_fullsized":"2024-09-15T21:23:33.646790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'11957","reported_seq":19048,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.703107+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:33.703107+0000","last_peered":"2024-09-15T21:23:33.703107+0000","last_clean":"2024-09-15T21:23:33.703107+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:33.703107+0000","last_undegraded":"2024-09-15T21:23:33.703107+0000","last_fullsized":"2024-09-15T21:23:33.703107+0000","mapping_epoch":47,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27284073,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437088897,"num_objects":106713,"num_object_clones":0,"num_object_copies":213426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161066,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12041,"num_bytes_recovered":41259008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010733056,"data_stored":1010669248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575088,"kb_used_data":322652,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93796752,"statfs":{"total":96636764160,"available":96047874048,"internally_reserved":0,"allocated":330395648,"data_stored":330110049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,1,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730780,"kb_used_data":374280,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93641060,"statfs":{"total":96636764160,"available":95888445440,"internally_reserved":0,"allocated":383262720,"data_stored":382986430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576420,"kb_used_data":292052,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93795420,"statfs":{"total":96636764160,"available":96046510080,"internally_reserved":0,"allocated":299061248,"data_stored":298725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382341120,"data_stored":382337641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298426368,"data_stored":298373590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329965568,"data_stored":329958017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:38.992 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:39.782 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:39.783 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:39.832 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T21:23:38.439239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437088916,"num_objects":106715,"num_object_clones":0,"num_object_copies":213430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161068,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12042,"num_bytes_recovered":41259027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1882288,"kb_used_data":988984,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":281233232,"statfs":{"total":289910292480,"available":287982829568,"internally_reserved":0,"allocated":1012719616,"data_stored":1011821701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":914672650},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,5,10,5,3,0,4,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8974336,"num_objects":-2191,"num_object_clones":0,"num_object_copies":-4382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1050,"num_objects_misplaced":750,"num_objects_unfound":0,"num_objects_dirty":-2191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2191,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":249856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001724"},"pg_stats":[{"pgid":"2.f","version":"48'11804","reported_seq":12379,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.477629+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:33.477629+0000","last_peered":"2024-09-15T21:23:33.477629+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:33.477629+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11804,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3588},{"shards":"0,2","objects":819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11922","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.531235+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:34.531235+0000","last_peered":"2024-09-15T21:23:34.531235+0000","last_clean":"2024-09-15T21:23:34.531235+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:34.531235+0000","last_undegraded":"2024-09-15T21:23:34.531235+0000","last_fullsized":"2024-09-15T21:23:34.531235+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10648,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12029","reported_seq":18949,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.734669+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:33.734669+0000","last_peered":"2024-09-15T21:23:33.734669+0000","last_clean":"2024-09-15T21:23:33.734669+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:33.734669+0000","last_undegraded":"2024-09-15T21:23:33.734669+0000","last_fullsized":"2024-09-15T21:23:33.734669+0000","mapping_epoch":45,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28016664,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10752,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'11949","reported_seq":12025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.527424+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:34.527424+0000","last_peered":"2024-09-15T21:23:34.527424+0000","last_clean":"2024-09-15T21:23:34.527424+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:34.527424+0000","last_undegraded":"2024-09-15T21:23:34.527424+0000","last_fullsized":"2024-09-15T21:23:34.527424+0000","mapping_epoch":36,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11949,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"48'11965","reported_seq":12088,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.463460+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:33.463460+0000","last_peered":"2024-09-15T21:23:33.463460+0000","last_clean":"2024-09-15T21:23:33.463460+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:33.463460+0000","last_undegraded":"2024-09-15T21:23:33.463460+0000","last_fullsized":"2024-09-15T21:23:33.463460+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27680768,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9823,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"48'11761","reported_seq":12435,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.518469+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:33.518469+0000","last_peered":"2024-09-15T21:23:33.518469+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:33.518469+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9186,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3519},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"48'12058","reported_seq":12636,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.460460+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:33.460460+0000","last_peered":"2024-09-15T21:23:33.460460+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:33.460460+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3600},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"48'11831","reported_seq":11955,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.481597+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:33.481597+0000","last_peered":"2024-09-15T21:23:33.481597+0000","last_clean":"2024-09-15T21:23:33.481597+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:33.481597+0000","last_undegraded":"2024-09-15T21:23:33.481597+0000","last_fullsized":"2024-09-15T21:23:33.481597+0000","mapping_epoch":35,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9616,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.645356+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:33.645356+0000","last_peered":"2024-09-15T21:23:33.645356+0000","last_clean":"2024-09-15T21:23:33.645356+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:33.645356+0000","last_undegraded":"2024-09-15T21:23:33.645356+0000","last_fullsized":"2024-09-15T21:23:33.645356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"48'11756","reported_seq":18570,"reported_epoch":49,"state":"peering","last_fresh":"2024-09-15T21:23:33.614574+0000","last_change":"2024-09-15T21:23:33.778706+0000","last_active":"2024-09-15T21:23:33.778706+0000","last_peered":"2024-09-15T21:23:33.614574+0000","last_clean":"2024-09-15T21:18:41.351933+0000","last_became_active":"2024-09-15T21:18:43.792774+0000","last_became_peered":"2024-09-15T21:18:43.792774+0000","last_unstale":"2024-09-15T21:23:33.614574+0000","last_undegraded":"2024-09-15T21:23:33.614574+0000","last_fullsized":"2024-09-15T21:18:43.369078+0000","mapping_epoch":49,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10342,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.639707+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:33.639707+0000","last_peered":"2024-09-15T21:23:33.639707+0000","last_clean":"2024-09-15T21:23:33.639707+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:33.639707+0000","last_undegraded":"2024-09-15T21:23:33.639707+0000","last_fullsized":"2024-09-15T21:23:33.639707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"48'11577","reported_seq":12196,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.493175+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:33.493175+0000","last_peered":"2024-09-15T21:23:33.493175+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:33.493175+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1500","ondisk_log_start":"37'1500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9103,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3362},{"shards":"0,2","objects":785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":105,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.557264+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:22:00.557264+0000","last_peered":"2024-09-15T21:22:00.557264+0000","last_clean":"2024-09-15T21:22:00.557264+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:22:00.557264+0000","last_undegraded":"2024-09-15T21:22:00.557264+0000","last_fullsized":"2024-09-15T21:22:00.557264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11662","reported_seq":12006,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.519204+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:34.519204+0000","last_peered":"2024-09-15T21:23:34.519204+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:34.519204+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27127808,"num_objects":6623,"num_object_clones":0,"num_object_copies":13246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":86,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641436+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:33.641436+0000","last_peered":"2024-09-15T21:23:33.641436+0000","last_clean":"2024-09-15T21:23:33.641436+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:33.641436+0000","last_undegraded":"2024-09-15T21:23:33.641436+0000","last_fullsized":"2024-09-15T21:23:33.641436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11878","reported_seq":12217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:34.443838+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:34.443838+0000","last_peered":"2024-09-15T21:23:34.443838+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:34.443838+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":3429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"48'11966","reported_seq":12555,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.480938+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:33.480938+0000","last_peered":"2024-09-15T21:23:33.480938+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:33.480938+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9273,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3539},{"shards":"0,2","objects":823}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.642773+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:33.642773+0000","last_peered":"2024-09-15T21:23:33.642773+0000","last_clean":"2024-09-15T21:23:33.642773+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:33.642773+0000","last_undegraded":"2024-09-15T21:23:33.642773+0000","last_fullsized":"2024-09-15T21:23:33.642773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'11779","reported_seq":12282,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.524327+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:33.524327+0000","last_peered":"2024-09-15T21:23:33.524327+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:33.524327+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27168768,"num_objects":6633,"num_object_clones":0,"num_object_copies":13266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":6633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9364,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.640890+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:33.640890+0000","last_peered":"2024-09-15T21:23:33.640890+0000","last_clean":"2024-09-15T21:23:33.640890+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:33.640890+0000","last_undegraded":"2024-09-15T21:23:33.640890+0000","last_fullsized":"2024-09-15T21:23:33.640890+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"48'11758","reported_seq":12310,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.410667+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:33.410667+0000","last_peered":"2024-09-15T21:23:33.410667+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:33.410667+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9405,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3467},{"shards":"0,2","objects":798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.641594+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:33.641594+0000","last_peered":"2024-09-15T21:23:33.641594+0000","last_clean":"2024-09-15T21:23:33.641594+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:33.641594+0000","last_undegraded":"2024-09-15T21:23:33.641594+0000","last_fullsized":"2024-09-15T21:23:33.641594+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.646790+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:33.646790+0000","last_peered":"2024-09-15T21:23:33.646790+0000","last_clean":"2024-09-15T21:23:33.646790+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:33.646790+0000","last_undegraded":"2024-09-15T21:23:33.646790+0000","last_fullsized":"2024-09-15T21:23:33.646790+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'11957","reported_seq":19048,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.703107+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:33.703107+0000","last_peered":"2024-09-15T21:23:33.703107+0000","last_clean":"2024-09-15T21:23:33.703107+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:33.703107+0000","last_undegraded":"2024-09-15T21:23:33.703107+0000","last_fullsized":"2024-09-15T21:23:33.703107+0000","mapping_epoch":47,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27284073,"num_objects":6662,"num_object_clones":0,"num_object_copies":13324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437088897,"num_objects":106713,"num_object_clones":0,"num_object_copies":213426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27715,"num_objects_misplaced":809,"num_objects_unfound":0,"num_objects_dirty":106713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161066,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12041,"num_bytes_recovered":41259008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010733056,"data_stored":1010669248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575088,"kb_used_data":322652,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93796752,"statfs":{"total":96636764160,"available":96047874048,"internally_reserved":0,"allocated":330395648,"data_stored":330110049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,1,1,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149775,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730780,"kb_used_data":374280,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93641060,"statfs":{"total":96636764160,"available":95888445440,"internally_reserved":0,"allocated":383262720,"data_stored":382986430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,3,2,0,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576420,"kb_used_data":292052,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93795420,"statfs":{"total":96636764160,"available":96046510080,"internally_reserved":0,"allocated":299061248,"data_stored":298725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":291172012},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382341120,"data_stored":382337641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298426368,"data_stored":298373590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329965568,"data_stored":329958017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "48'11804", 'reported_seq': 12379, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.477629+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:33.477629+0000', 'last_peered': '2024-09-15T21:23:33.477629+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:33.477629+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27475968, 'num_objects': 6708, 'num_object_clones': 0, 'num_object_copies': 13416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11804, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 247, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3588}, {'shards': '0,2', 'objects': 819}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "48'11761", 'reported_seq': 12435, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.518469+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:33.518469+0000', 'last_peered': '2024-09-15T21:23:33.518469+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:33.518469+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27115520, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9186, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 270, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3519}, {'shards': '0,2', 'objects': 803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "48'12058", 'reported_seq': 12636, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.460460+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:33.460460+0000', 'last_peered': '2024-09-15T21:23:33.460460+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:33.460460+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28041216, 'num_objects': 6846, 'num_object_clones': 0, 'num_object_copies': 13692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9694, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 225, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3600}, {'shards': '0,2', 'objects': 866}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:39.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "48'11756", 'reported_seq': 18570, 'reported_epoch': 49, 'state': 'peering', 'last_fresh': '2024-09-15T21:23:33.614574+0000', 'last_change': '2024-09-15T21:23:33.778706+0000', 'last_active': '2024-09-15T21:23:33.778706+0000', 'last_peered': '2024-09-15T21:23:33.614574+0000', 'last_clean': '2024-09-15T21:18:41.351933+0000', 'last_became_active': '2024-09-15T21:18:43.792774+0000', 'last_became_peered': '2024-09-15T21:18:43.792774+0000', 'last_unstale': '2024-09-15T21:23:33.614574+0000', 'last_undegraded': '2024-09-15T21:23:33.614574+0000', 'last_fullsized': '2024-09-15T21:18:43.369078+0000', 'mapping_epoch': 49, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1431", 'last_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:08.359695+0000', 'objects_scrubbed': 1407, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 575, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27111424, 'num_objects': 6619, 'num_object_clones': 0, 'num_object_copies': 13238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10342, 'num_write_kb': 31108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3340, 'num_bytes_recovered': 13144064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "48'11577", 'reported_seq': 12196, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.493175+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:33.493175+0000', 'last_peered': '2024-09-15T21:23:33.493175+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:33.493175+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1500", 'ondisk_log_start': "37'1500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26447872, 'num_objects': 6457, 'num_object_clones': 0, 'num_object_copies': 12914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9103, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 246, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3362}, {'shards': '0,2', 'objects': 785}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'11662", 'reported_seq': 12006, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:34.519204+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:34.519204+0000', 'last_peered': '2024-09-15T21:23:34.519204+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:34.519204+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1600", 'ondisk_log_start': "37'1600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27127808, 'num_objects': 6623, 'num_object_clones': 0, 'num_object_copies': 13246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10213, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 114, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 749}, {'shards': '2', 'objects': 3471}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "49'11878", 'reported_seq': 12217, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:34.443838+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:34.443838+0000', 'last_peered': '2024-09-15T21:23:34.443838+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:34.443838+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27320320, 'num_objects': 6670, 'num_object_clones': 0, 'num_object_copies': 13340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3243, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6670, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10277, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 109, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 749}, {'shards': '2', 'objects': 3429}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "48'11966", 'reported_seq': 12555, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.480938+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:33.480938+0000', 'last_peered': '2024-09-15T21:23:33.480938+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:33.480938+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27623424, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9273, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 228, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3539}, {'shards': '0,2', 'objects': 823}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "48'11779", 'reported_seq': 12282, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.524327+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:33.524327+0000', 'last_peered': '2024-09-15T21:23:33.524327+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:33.524327+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27168768, 'num_objects': 6633, 'num_object_clones': 0, 'num_object_copies': 13266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2389, 'num_objects_misplaced': 809, 'num_objects_unfound': 0, 'num_objects_dirty': 6633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9364, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 184, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3435}, {'shards': '0,2', 'objects': 809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:39.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "48'11758", 'reported_seq': 12310, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.410667+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:33.410667+0000', 'last_peered': '2024-09-15T21:23:33.410667+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:33.410667+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27115520, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9405, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 212, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3467}, {'shards': '0,2', 'objects': 798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:39.836 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:40.682 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:40.682 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27715/213430 objects degraded (12.986%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":433,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106715,"data_bytes":437088916,"bytes_used":1927462912,"bytes_avail":287982829568,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":27715,"degraded_total":213430,"degraded_ratio":0.12985522185259804,"misplaced_objects":809,"misplaced_total":213430,"misplaced_ratio":0.0037904699433069389,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":31225,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":249856,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":273},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:40.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:40.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:41.430 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:41.489 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-15T21:23:40.439657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":428479124,"num_objects":104613,"num_object_clones":0,"num_object_copies":209226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27646,"num_objects_misplaced":630,"num_objects_unfound":0,"num_objects_dirty":104613,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163170,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12290,"num_bytes_recovered":41992211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161057,"ondisk_log_size":161057,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1905260,"kb_used_data":977256,"kb_used_omap":12,"kb_used_meta":927923,"kb_avail":281210260,"statfs":{"total":289910292480,"available":287959306240,"internally_reserved":0,"allocated":1000710144,"data_stored":999796418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":950193162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,5,2,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17584128,"num_objects":-4293,"num_object_clones":0,"num_object_copies":-8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1119,"num_objects_misplaced":571,"num_objects_unfound":0,"num_objects_dirty":-4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4293,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":548,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001613"},"pg_stats":[{"pgid":"2.f","version":"50'11936","reported_seq":12539,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.504169+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:38.504169+0000","last_peered":"2024-09-15T21:23:38.504169+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:38.504169+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11936,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3468},{"shards":"0,2","objects":687}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12039","reported_seq":12144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.524476+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:39.524476+0000","last_peered":"2024-09-15T21:23:39.524476+0000","last_clean":"2024-09-15T21:23:39.524476+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:39.524476+0000","last_undegraded":"2024-09-15T21:23:39.524476+0000","last_fullsized":"2024-09-15T21:23:39.524476+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12164","reported_seq":19088,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.792958+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:38.792958+0000","last_peered":"2024-09-15T21:23:38.792958+0000","last_clean":"2024-09-15T21:23:38.792958+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:38.792958+0000","last_undegraded":"2024-09-15T21:23:38.792958+0000","last_fullsized":"2024-09-15T21:23:38.792958+0000","mapping_epoch":45,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27463704,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10887,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12080","reported_seq":12158,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.513440+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:39.513440+0000","last_peered":"2024-09-15T21:23:39.513440+0000","last_clean":"2024-09-15T21:23:39.513440+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:39.513440+0000","last_undegraded":"2024-09-15T21:23:39.513440+0000","last_fullsized":"2024-09-15T21:23:39.513440+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26583040,"num_objects":6490,"num_object_clones":0,"num_object_copies":12980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12080,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12110","reported_seq":12237,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.568181+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:38.568181+0000","last_peered":"2024-09-15T21:23:38.568181+0000","last_clean":"2024-09-15T21:23:38.568181+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:38.568181+0000","last_undegraded":"2024-09-15T21:23:38.568181+0000","last_fullsized":"2024-09-15T21:23:38.568181+0000","mapping_epoch":35,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27086848,"num_objects":6613,"num_object_clones":0,"num_object_copies":13226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9968,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'11894","reported_seq":12588,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.642555+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:38.642555+0000","last_peered":"2024-09-15T21:23:38.642555+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:38.642555+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9319,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3394},{"shards":"0,2","objects":670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12219","reported_seq":12815,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.410134+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:38.410134+0000","last_peered":"2024-09-15T21:23:38.410134+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:38.410134+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381760,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9855,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3446},{"shards":"0,2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'11961","reported_seq":12089,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.605973+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:38.605973+0000","last_peered":"2024-09-15T21:23:38.605973+0000","last_clean":"2024-09-15T21:23:38.605973+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:38.605973+0000","last_undegraded":"2024-09-15T21:23:38.605973+0000","last_fullsized":"2024-09-15T21:23:38.605973+0000","mapping_epoch":35,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26390528,"num_objects":6443,"num_object_clones":0,"num_object_copies":12886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'11880","reported_seq":18703,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.764501+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:38.764501+0000","last_peered":"2024-09-15T21:23:38.764501+0000","last_clean":"2024-09-15T21:23:38.764501+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:38.764501+0000","last_undegraded":"2024-09-15T21:23:38.764501+0000","last_fullsized":"2024-09-15T21:23:38.764501+0000","mapping_epoch":49,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26603520,"num_objects":6495,"num_object_clones":0,"num_object_copies":12990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10466,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11700","reported_seq":12342,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.620160+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:38.620160+0000","last_peered":"2024-09-15T21:23:38.620160+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:38.620160+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1600","ondisk_log_start":"37'1600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9226,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'11783","reported_seq":12143,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.456964+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:39.456964+0000","last_peered":"2024-09-15T21:23:39.456964+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:39.456964+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10334,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'11999","reported_seq":12348,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.551862+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:39.551862+0000","last_peered":"2024-09-15T21:23:39.551862+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:39.551862+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10398,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12111","reported_seq":12740,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.673808+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:38.673808+0000","last_peered":"2024-09-15T21:23:38.673808+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:38.673808+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27029504,"num_objects":6599,"num_object_clones":0,"num_object_copies":13198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9418,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3412},{"shards":"0,2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'11899","reported_seq":12764,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.633958+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:38.633958+0000","last_peered":"2024-09-15T21:23:38.633958+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:38.633958+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26677248,"num_objects":6513,"num_object_clones":0,"num_object_copies":13026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":630,"num_objects_unfound":0,"num_objects_dirty":6513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9484,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":363,"num_bytes_recovered":741376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3494},{"shards":"0,2","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'11884","reported_seq":12448,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.610953+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:38.610953+0000","last_peered":"2024-09-15T21:23:38.610953+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:38.610953+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26599424,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9531,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3345},{"shards":"0,2","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12095","reported_seq":19188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.728863+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:38.728863+0000","last_peered":"2024-09-15T21:23:38.728863+0000","last_clean":"2024-09-15T21:23:38.728863+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:38.728863+0000","last_undegraded":"2024-09-15T21:23:38.728863+0000","last_fullsized":"2024-09-15T21:23:38.728863+0000","mapping_epoch":47,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718825,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9755,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":428479105,"num_objects":104611,"num_object_clones":0,"num_object_copies":209222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27646,"num_objects_misplaced":630,"num_objects_unfound":0,"num_objects_dirty":104611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163168,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12289,"num_bytes_recovered":41992192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":998395904,"data_stored":998332096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575060,"kb_used_data":318468,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93796780,"statfs":{"total":96636764160,"available":96047902720,"internally_reserved":0,"allocated":326111232,"data_stored":325820375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149777,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732832,"kb_used_data":368000,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93639008,"statfs":{"total":96636764160,"available":95886344192,"internally_reserved":0,"allocated":376832000,"data_stored":376545215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":373550769},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597368,"kb_used_data":290788,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93774472,"statfs":{"total":96636764160,"available":96025059328,"internally_reserved":0,"allocated":297766912,"data_stored":297430828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":375779328,"data_stored":375775849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296988672,"data_stored":296935894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325627904,"data_stored":325620353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:44.492 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:45.037 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:45.056 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-15T21:23:44.443129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422228628,"num_objects":103087,"num_object_clones":0,"num_object_copies":206174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27566,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":103087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164696,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12521,"num_bytes_recovered":42610707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1876840,"kb_used_data":969656,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":281238680,"statfs":{"total":289910292480,"available":287988408320,"internally_reserved":0,"allocated":992927744,"data_stored":992014018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":928893962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,4,3,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17137664,"num_objects":-4184,"num_object_clones":0,"num_object_copies":-8368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":-330,"num_objects_unfound":0,"num_objects_dirty":-4184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4184,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":1351680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004660"},"pg_stats":[{"pgid":"2.f","version":"50'12075","reported_seq":12700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.655428+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:43.655428+0000","last_peered":"2024-09-15T21:23:43.655428+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:43.655428+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12075,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3340},{"shards":"0,2","objects":548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12039","reported_seq":12144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.524476+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:39.524476+0000","last_peered":"2024-09-15T21:23:39.524476+0000","last_clean":"2024-09-15T21:23:39.524476+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:39.524476+0000","last_undegraded":"2024-09-15T21:23:39.524476+0000","last_fullsized":"2024-09-15T21:23:39.524476+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12311","reported_seq":19236,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.797407+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:43.797407+0000","last_peered":"2024-09-15T21:23:43.797407+0000","last_clean":"2024-09-15T21:23:43.797407+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:43.797407+0000","last_undegraded":"2024-09-15T21:23:43.797407+0000","last_fullsized":"2024-09-15T21:23:43.797407+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861592,"num_objects":6559,"num_object_clones":0,"num_object_copies":13118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11034,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12080","reported_seq":12158,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.513440+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:39.513440+0000","last_peered":"2024-09-15T21:23:39.513440+0000","last_clean":"2024-09-15T21:23:39.513440+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:39.513440+0000","last_undegraded":"2024-09-15T21:23:39.513440+0000","last_fullsized":"2024-09-15T21:23:39.513440+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26583040,"num_objects":6490,"num_object_clones":0,"num_object_copies":12980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12080,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12215","reported_seq":12342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.608580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:43.608580+0000","last_peered":"2024-09-15T21:23:43.608580+0000","last_clean":"2024-09-15T21:23:43.608580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:43.608580+0000","last_undegraded":"2024-09-15T21:23:43.608580+0000","last_fullsized":"2024-09-15T21:23:43.608580+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10073,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12023","reported_seq":12743,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.542504+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:43.542504+0000","last_peered":"2024-09-15T21:23:43.542504+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:43.542504+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3278},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12348","reported_seq":12972,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.687388+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:43.687388+0000","last_peered":"2024-09-15T21:23:43.687388+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:43.687388+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9984,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12084","reported_seq":12212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.657382+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:43.657382+0000","last_peered":"2024-09-15T21:23:43.657382+0000","last_clean":"2024-09-15T21:23:43.657382+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:43.657382+0000","last_undegraded":"2024-09-15T21:23:43.657382+0000","last_fullsized":"2024-09-15T21:23:43.657382+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9869,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'11989","reported_seq":18812,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.769695+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:43.769695+0000","last_peered":"2024-09-15T21:23:43.769695+0000","last_clean":"2024-09-15T21:23:43.769695+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:43.769695+0000","last_undegraded":"2024-09-15T21:23:43.769695+0000","last_fullsized":"2024-09-15T21:23:43.769695+0000","mapping_epoch":49,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26157056,"num_objects":6386,"num_object_clones":0,"num_object_copies":12772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10575,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11813","reported_seq":12491,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.672817+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:43.672817+0000","last_peered":"2024-09-15T21:23:43.672817+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:43.672817+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25481216,"num_objects":6221,"num_object_clones":0,"num_object_copies":12442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9339,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3154},{"shards":"0,2","objects":549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'11783","reported_seq":12143,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.456964+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:39.456964+0000","last_peered":"2024-09-15T21:23:39.456964+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:39.456964+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10334,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'11999","reported_seq":12348,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.551862+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:39.551862+0000","last_peered":"2024-09-15T21:23:39.551862+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:39.551862+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10398,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12234","reported_seq":12875,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.696085+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:43.696085+0000","last_peered":"2024-09-15T21:23:43.696085+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:43.696085+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26525696,"num_objects":6476,"num_object_clones":0,"num_object_copies":12952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9541,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3295},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12032","reported_seq":13199,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.699655+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:43.699655+0000","last_peered":"2024-09-15T21:23:43.699655+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:43.699655+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3512},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12028","reported_seq":12624,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.701148+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:43.701148+0000","last_peered":"2024-09-15T21:23:43.701148+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:43.701148+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26009600,"num_objects":6350,"num_object_clones":0,"num_object_copies":12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3217},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12227","reported_seq":19320,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.791167+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:43.791167+0000","last_peered":"2024-09-15T21:23:43.791167+0000","last_clean":"2024-09-15T21:23:43.791167+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:43.791167+0000","last_undegraded":"2024-09-15T21:23:43.791167+0000","last_fullsized":"2024-09-15T21:23:43.791167+0000","mapping_epoch":47,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26178153,"num_objects":6392,"num_object_clones":0,"num_object_copies":12784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9887,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422228609,"num_objects":103085,"num_object_clones":0,"num_object_copies":206170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27566,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":103085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164694,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12520,"num_bytes_recovered":42610688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990576640,"data_stored":990512832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575060,"kb_used_data":318468,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93796780,"statfs":{"total":96636764160,"available":96047902720,"internally_reserved":0,"allocated":326111232,"data_stored":325820375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714288,"kb_used_data":361952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93657552,"statfs":{"total":96636764160,"available":95905333248,"internally_reserved":0,"allocated":370638848,"data_stored":370352063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587492,"kb_used_data":289236,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93784348,"statfs":{"total":96636764160,"available":96035172352,"internally_reserved":0,"allocated":296177664,"data_stored":295841580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369438720,"data_stored":369435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295510016,"data_stored":295457238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325627904,"data_stored":325620353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:45.058 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:45.679 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:45.744 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-15T21:23:44.443129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422228628,"num_objects":103087,"num_object_clones":0,"num_object_copies":206174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27566,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":103087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164696,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12521,"num_bytes_recovered":42610707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1876840,"kb_used_data":969656,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":281238680,"statfs":{"total":289910292480,"available":287988408320,"internally_reserved":0,"allocated":992927744,"data_stored":992014018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":928893962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,4,3,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17137664,"num_objects":-4184,"num_object_clones":0,"num_object_copies":-8368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":-330,"num_objects_unfound":0,"num_objects_dirty":-4184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4184,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":1351680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004660"},"pg_stats":[{"pgid":"2.f","version":"50'12075","reported_seq":12700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.655428+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:43.655428+0000","last_peered":"2024-09-15T21:23:43.655428+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:43.655428+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12075,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3340},{"shards":"0,2","objects":548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12039","reported_seq":12144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.524476+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:39.524476+0000","last_peered":"2024-09-15T21:23:39.524476+0000","last_clean":"2024-09-15T21:23:39.524476+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:39.524476+0000","last_undegraded":"2024-09-15T21:23:39.524476+0000","last_fullsized":"2024-09-15T21:23:39.524476+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27037696,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10765,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12311","reported_seq":19236,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.797407+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:43.797407+0000","last_peered":"2024-09-15T21:23:43.797407+0000","last_clean":"2024-09-15T21:23:43.797407+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:43.797407+0000","last_undegraded":"2024-09-15T21:23:43.797407+0000","last_fullsized":"2024-09-15T21:23:43.797407+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861592,"num_objects":6559,"num_object_clones":0,"num_object_copies":13118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11034,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12080","reported_seq":12158,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:39.513440+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:39.513440+0000","last_peered":"2024-09-15T21:23:39.513440+0000","last_clean":"2024-09-15T21:23:39.513440+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:39.513440+0000","last_undegraded":"2024-09-15T21:23:39.513440+0000","last_fullsized":"2024-09-15T21:23:39.513440+0000","mapping_epoch":36,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26583040,"num_objects":6490,"num_object_clones":0,"num_object_copies":12980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12080,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12215","reported_seq":12342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.608580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:43.608580+0000","last_peered":"2024-09-15T21:23:43.608580+0000","last_clean":"2024-09-15T21:23:43.608580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:43.608580+0000","last_undegraded":"2024-09-15T21:23:43.608580+0000","last_fullsized":"2024-09-15T21:23:43.608580+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10073,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12023","reported_seq":12743,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.542504+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:43.542504+0000","last_peered":"2024-09-15T21:23:43.542504+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:43.542504+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3278},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12348","reported_seq":12972,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.687388+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:43.687388+0000","last_peered":"2024-09-15T21:23:43.687388+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:43.687388+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9984,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12084","reported_seq":12212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.657382+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:43.657382+0000","last_peered":"2024-09-15T21:23:43.657382+0000","last_clean":"2024-09-15T21:23:43.657382+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:43.657382+0000","last_undegraded":"2024-09-15T21:23:43.657382+0000","last_fullsized":"2024-09-15T21:23:43.657382+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9869,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'11989","reported_seq":18812,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.769695+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:43.769695+0000","last_peered":"2024-09-15T21:23:43.769695+0000","last_clean":"2024-09-15T21:23:43.769695+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:43.769695+0000","last_undegraded":"2024-09-15T21:23:43.769695+0000","last_fullsized":"2024-09-15T21:23:43.769695+0000","mapping_epoch":49,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26157056,"num_objects":6386,"num_object_clones":0,"num_object_copies":12772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10575,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11813","reported_seq":12491,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.672817+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:43.672817+0000","last_peered":"2024-09-15T21:23:43.672817+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:43.672817+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25481216,"num_objects":6221,"num_object_clones":0,"num_object_copies":12442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9339,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3154},{"shards":"0,2","objects":549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'11783","reported_seq":12143,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.456964+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:39.456964+0000","last_peered":"2024-09-15T21:23:39.456964+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:39.456964+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1700","ondisk_log_start":"37'1700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10334,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3357}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'11999","reported_seq":12348,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:39.551862+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:39.551862+0000","last_peered":"2024-09-15T21:23:39.551862+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:39.551862+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10398,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":628},{"shards":"2","objects":3312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12234","reported_seq":12875,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.696085+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:43.696085+0000","last_peered":"2024-09-15T21:23:43.696085+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:43.696085+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26525696,"num_objects":6476,"num_object_clones":0,"num_object_copies":12952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9541,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3295},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12032","reported_seq":13199,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.699655+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:43.699655+0000","last_peered":"2024-09-15T21:23:43.699655+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:43.699655+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3512},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12028","reported_seq":12624,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.701148+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:43.701148+0000","last_peered":"2024-09-15T21:23:43.701148+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:43.701148+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26009600,"num_objects":6350,"num_object_clones":0,"num_object_copies":12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3217},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12227","reported_seq":19320,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.791167+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:43.791167+0000","last_peered":"2024-09-15T21:23:43.791167+0000","last_clean":"2024-09-15T21:23:43.791167+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:43.791167+0000","last_undegraded":"2024-09-15T21:23:43.791167+0000","last_fullsized":"2024-09-15T21:23:43.791167+0000","mapping_epoch":47,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26178153,"num_objects":6392,"num_object_clones":0,"num_object_copies":12784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9887,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422228609,"num_objects":103085,"num_object_clones":0,"num_object_copies":206170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27566,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":103085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164694,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12520,"num_bytes_recovered":42610688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":990576640,"data_stored":990512832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953549,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575060,"kb_used_data":318468,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93796780,"statfs":{"total":96636764160,"available":96047902720,"internally_reserved":0,"allocated":326111232,"data_stored":325820375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714288,"kb_used_data":361952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93657552,"statfs":{"total":96636764160,"available":95905333248,"internally_reserved":0,"allocated":370638848,"data_stored":370352063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587492,"kb_used_data":289236,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93784348,"statfs":{"total":96636764160,"available":96035172352,"internally_reserved":0,"allocated":296177664,"data_stored":295841580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369438720,"data_stored":369435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295510016,"data_stored":295457238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":325627904,"data_stored":325620353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12075", 'reported_seq': 12700, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.655428+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:43.655428+0000', 'last_peered': '2024-09-15T21:23:43.655428+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:43.655428+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26365952, 'num_objects': 6437, 'num_object_clones': 0, 'num_object_copies': 12874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12075, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 270, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3340}, {'shards': '0,2', 'objects': 548}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12023", 'reported_seq': 12743, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.542504+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:43.542504+0000', 'last_peered': '2024-09-15T21:23:43.542504+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:43.542504+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26042368, 'num_objects': 6358, 'num_object_clones': 0, 'num_object_copies': 12716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3082, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9448, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 291, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3278}, {'shards': '0,2', 'objects': 541}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'12348", 'reported_seq': 12972, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.687388+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:43.687388+0000', 'last_peered': '2024-09-15T21:23:43.687388+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:43.687388+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26853376, 'num_objects': 6556, 'num_object_clones': 0, 'num_object_copies': 13112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3226, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9984, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 247, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3332}, {'shards': '0,2', 'objects': 576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:45.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'11813", 'reported_seq': 12491, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.672817+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:43.672817+0000', 'last_peered': '2024-09-15T21:23:43.672817+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:43.672817+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1800", 'ondisk_log_start': "37'1800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25481216, 'num_objects': 6221, 'num_object_clones': 0, 'num_object_copies': 12442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3069, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9339, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 274, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3154}, {'shards': '0,2', 'objects': 549}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'11783", 'reported_seq': 12143, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:39.456964+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:39.456964+0000', 'last_peered': '2024-09-15T21:23:39.456964+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:39.456964+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'1700", 'ondisk_log_start': "37'1700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26632192, 'num_objects': 6502, 'num_object_clones': 0, 'num_object_copies': 13004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10334, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 121, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 628}, {'shards': '2', 'objects': 3357}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'11999", 'reported_seq': 12348, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:39.551862+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:39.551862+0000', 'last_peered': '2024-09-15T21:23:39.551862+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:39.551862+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26824704, 'num_objects': 6549, 'num_object_clones': 0, 'num_object_copies': 13098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3239, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10398, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 113, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 628}, {'shards': '2', 'objects': 3312}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'12234", 'reported_seq': 12875, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.696085+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:43.696085+0000', 'last_peered': '2024-09-15T21:23:43.696085+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:43.696085+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26525696, 'num_objects': 6476, 'num_object_clones': 0, 'num_object_copies': 12952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3182, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9541, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 252, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3295}, {'shards': '0,2', 'objects': 555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12032", 'reported_seq': 13199, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.699655+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:43.699655+0000', 'last_peered': '2024-09-15T21:23:43.699655+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:43.699655+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26132480, 'num_objects': 6380, 'num_object_clones': 0, 'num_object_copies': 12760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2389, 'num_objects_misplaced': 479, 'num_objects_unfound': 0, 'num_objects_dirty': 6380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9617, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 514, 'num_bytes_recovered': 1359872, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3512}, {'shards': '0,2', 'objects': 479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:45.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12028", 'reported_seq': 12624, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.701148+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:43.701148+0000', 'last_peered': '2024-09-15T21:23:43.701148+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:43.701148+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26009600, 'num_objects': 6350, 'num_object_clones': 0, 'num_object_copies': 12700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9675, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3217}, {'shards': '0,2', 'objects': 528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:46.562 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:46.562 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27566/206174 objects degraded (13.370%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":439,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103087,"data_bytes":422228628,"bytes_used":1921884160,"bytes_avail":287988408320,"bytes_total":289910292480,"degraded_objects":27566,"degraded_total":206174,"degraded_ratio":0.13370260071590015,"misplaced_objects":479,"misplaced_total":206174,"misplaced_ratio":0.0023232803360268509,"recovering_objects_per_sec":49,"recovering_bytes_per_sec":135105,"recovering_keys_per_sec":0,"num_objects_recovered":494,"num_bytes_recovered":1351680,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":418},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:46.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:46.619 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:47.320 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:47.343 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-15T21:23:46.443640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420012692,"num_objects":102546,"num_object_clones":0,"num_object_copies":205092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27555,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":102546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165237,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12532,"num_bytes_recovered":42610707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160524,"ondisk_log_size":160524,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1872888,"kb_used_data":965704,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":281242632,"statfs":{"total":289910292480,"available":287992455168,"internally_reserved":0,"allocated":988880896,"data_stored":987967170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":928893962},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,2,4,2,12,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17076224,"num_objects":-4169,"num_object_clones":0,"num_object_copies":-8338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":-330,"num_objects_unfound":0,"num_objects_dirty":-4169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4169,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":490,"num_bytes_recovered":1351680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.774949"},"pg_stats":[{"pgid":"2.f","version":"50'12075","reported_seq":12700,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.655428+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:43.655428+0000","last_peered":"2024-09-15T21:23:43.655428+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:43.655428+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12075,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3340},{"shards":"0,2","objects":548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12201","reported_seq":12304,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:44.581313+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:44.581313+0000","last_peered":"2024-09-15T21:23:44.581313+0000","last_clean":"2024-09-15T21:23:44.581313+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:44.581313+0000","last_undegraded":"2024-09-15T21:23:44.581313+0000","last_fullsized":"2024-09-15T21:23:44.581313+0000","mapping_epoch":36,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26374144,"num_objects":6439,"num_object_clones":0,"num_object_copies":12878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10927,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12311","reported_seq":19236,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.797407+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:43.797407+0000","last_peered":"2024-09-15T21:23:43.797407+0000","last_clean":"2024-09-15T21:23:43.797407+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:43.797407+0000","last_undegraded":"2024-09-15T21:23:43.797407+0000","last_fullsized":"2024-09-15T21:23:43.797407+0000","mapping_epoch":45,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26861592,"num_objects":6559,"num_object_clones":0,"num_object_copies":13118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11034,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12214","reported_seq":12292,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:44.518769+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:44.518769+0000","last_peered":"2024-09-15T21:23:44.518769+0000","last_clean":"2024-09-15T21:23:44.518769+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:44.518769+0000","last_undegraded":"2024-09-15T21:23:44.518769+0000","last_fullsized":"2024-09-15T21:23:44.518769+0000","mapping_epoch":36,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26034176,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12214,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12215","reported_seq":12342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.608580+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:43.608580+0000","last_peered":"2024-09-15T21:23:43.608580+0000","last_clean":"2024-09-15T21:23:43.608580+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:43.608580+0000","last_undegraded":"2024-09-15T21:23:43.608580+0000","last_fullsized":"2024-09-15T21:23:43.608580+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10073,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12023","reported_seq":12743,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.542504+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:43.542504+0000","last_peered":"2024-09-15T21:23:43.542504+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:43.542504+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3278},{"shards":"0,2","objects":541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12348","reported_seq":12972,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.687388+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:43.687388+0000","last_peered":"2024-09-15T21:23:43.687388+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:43.687388+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9984,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12084","reported_seq":12212,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.657382+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:43.657382+0000","last_peered":"2024-09-15T21:23:43.657382+0000","last_clean":"2024-09-15T21:23:43.657382+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:43.657382+0000","last_undegraded":"2024-09-15T21:23:43.657382+0000","last_fullsized":"2024-09-15T21:23:43.657382+0000","mapping_epoch":35,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9869,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'11989","reported_seq":18812,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.769695+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:43.769695+0000","last_peered":"2024-09-15T21:23:43.769695+0000","last_clean":"2024-09-15T21:23:43.769695+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:43.769695+0000","last_undegraded":"2024-09-15T21:23:43.769695+0000","last_fullsized":"2024-09-15T21:23:43.769695+0000","mapping_epoch":49,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26157056,"num_objects":6386,"num_object_clones":0,"num_object_copies":12772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10575,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11813","reported_seq":12491,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.672817+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:43.672817+0000","last_peered":"2024-09-15T21:23:43.672817+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:43.672817+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1800","ondisk_log_start":"37'1800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25481216,"num_objects":6221,"num_object_clones":0,"num_object_copies":12442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9339,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3154},{"shards":"0,2","objects":549}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'11915","reported_seq":12293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:44.568205+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:44.568205+0000","last_peered":"2024-09-15T21:23:44.568205+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:44.568205+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26091520,"num_objects":6370,"num_object_clones":0,"num_object_copies":12740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3138,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10466,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":496},{"shards":"2","objects":3234}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12112","reported_seq":12466,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:44.549285+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:44.549285+0000","last_peered":"2024-09-15T21:23:44.549285+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:44.549285+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26361856,"num_objects":6436,"num_object_clones":0,"num_object_copies":12872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10511,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":515},{"shards":"2","objects":3201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12234","reported_seq":12875,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.696085+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:43.696085+0000","last_peered":"2024-09-15T21:23:43.696085+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:43.696085+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26525696,"num_objects":6476,"num_object_clones":0,"num_object_copies":12952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9541,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3295},{"shards":"0,2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12032","reported_seq":13199,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.699655+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:43.699655+0000","last_peered":"2024-09-15T21:23:43.699655+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:43.699655+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26132480,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9617,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3512},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12028","reported_seq":12624,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.701148+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:43.701148+0000","last_peered":"2024-09-15T21:23:43.701148+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:43.701148+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26009600,"num_objects":6350,"num_object_clones":0,"num_object_copies":12700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3217},{"shards":"0,2","objects":528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12227","reported_seq":19320,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.791167+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:43.791167+0000","last_peered":"2024-09-15T21:23:43.791167+0000","last_clean":"2024-09-15T21:23:43.791167+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:43.791167+0000","last_undegraded":"2024-09-15T21:23:43.791167+0000","last_fullsized":"2024-09-15T21:23:43.791167+0000","mapping_epoch":47,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26178153,"num_objects":6392,"num_object_clones":0,"num_object_copies":12784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9887,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420012673,"num_objects":102544,"num_object_clones":0,"num_object_copies":205088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27555,"num_objects_misplaced":479,"num_objects_unfound":0,"num_objects_dirty":102544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165235,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12531,"num_bytes_recovered":42610688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":986382336,"data_stored":986318528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160521,"ondisk_log_size":160521,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953550,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571108,"kb_used_data":314516,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93800732,"statfs":{"total":96636764160,"available":96051949568,"internally_reserved":0,"allocated":322064384,"data_stored":321773527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,3,1,1,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714288,"kb_used_data":361952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93657552,"statfs":{"total":96636764160,"available":95905333248,"internally_reserved":0,"allocated":370638848,"data_stored":370352063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626496,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587492,"kb_used_data":289236,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93784348,"statfs":{"total":96636764160,"available":96035172352,"internally_reserved":0,"allocated":296177664,"data_stored":295841580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":369438720,"data_stored":369435241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295510016,"data_stored":295457238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321433600,"data_stored":321426049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:50.346 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:50.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:50.992 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:51.034 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-15T21:23:50.448499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":411800212,"num_objects":100541,"num_object_clones":0,"num_object_copies":201082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167242,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12788,"num_bytes_recovered":43290643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857480,"kb_used_data":954452,"kb_used_omap":12,"kb_used_meta":902963,"kb_avail":281258040,"statfs":{"total":289910292480,"available":288008232960,"internally_reserved":0,"allocated":977358848,"data_stored":976445122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":924634122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,7,9,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25288704,"num_objects":-6174,"num_object_clones":0,"num_object_copies":-12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":-496,"num_objects_unfound":0,"num_objects_dirty":-6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":746,"num_bytes_recovered":2031616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009259"},"pg_stats":[{"pgid":"2.f","version":"50'12213","reported_seq":12874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.701710+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:48.701710+0000","last_peered":"2024-09-15T21:23:48.701710+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:48.701710+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25800704,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12331","reported_seq":12436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.403812+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:49.403812+0000","last_peered":"2024-09-15T21:23:49.403812+0000","last_clean":"2024-09-15T21:23:49.403812+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:49.403812+0000","last_undegraded":"2024-09-15T21:23:49.403812+0000","last_fullsized":"2024-09-15T21:23:49.403812+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12417","reported_seq":19342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.634949+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:48.634949+0000","last_peered":"2024-09-15T21:23:48.634949+0000","last_clean":"2024-09-15T21:23:48.634949+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:48.634949+0000","last_undegraded":"2024-09-15T21:23:48.634949+0000","last_fullsized":"2024-09-15T21:23:48.634949+0000","mapping_epoch":45,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26427416,"num_objects":6453,"num_object_clones":0,"num_object_copies":12906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11140,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12334","reported_seq":12412,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.377994+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:49.377994+0000","last_peered":"2024-09-15T21:23:49.377994+0000","last_clean":"2024-09-15T21:23:49.377994+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:49.377994+0000","last_undegraded":"2024-09-15T21:23:49.377994+0000","last_fullsized":"2024-09-15T21:23:49.377994+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12349","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.630149+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:48.630149+0000","last_peered":"2024-09-15T21:23:48.630149+0000","last_clean":"2024-09-15T21:23:48.630149+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:48.630149+0000","last_undegraded":"2024-09-15T21:23:48.630149+0000","last_fullsized":"2024-09-15T21:23:48.630149+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26107904,"num_objects":6374,"num_object_clones":0,"num_object_copies":12748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10207,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12157","reported_seq":12910,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.703568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:48.703568+0000","last_peered":"2024-09-15T21:23:48.703568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:48.703568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12496","reported_seq":13154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.600563+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:48.600563+0000","last_peered":"2024-09-15T21:23:48.600563+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:48.600563+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3200},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12216","reported_seq":12344,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.696337+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:48.696337+0000","last_peered":"2024-09-15T21:23:48.696337+0000","last_clean":"2024-09-15T21:23:48.696337+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:48.696337+0000","last_undegraded":"2024-09-15T21:23:48.696337+0000","last_fullsized":"2024-09-15T21:23:48.696337+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10001,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12115","reported_seq":18938,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.758256+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:48.758256+0000","last_peered":"2024-09-15T21:23:48.758256+0000","last_clean":"2024-09-15T21:23:48.758256+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:48.758256+0000","last_undegraded":"2024-09-15T21:23:48.758256+0000","last_fullsized":"2024-09-15T21:23:48.758256+0000","mapping_epoch":49,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10701,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11929","reported_seq":12631,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.666935+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:48.666935+0000","last_peered":"2024-09-15T21:23:48.666935+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:48.666935+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9455,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12034","reported_seq":12418,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.343804+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:49.343804+0000","last_peered":"2024-09-15T21:23:49.343804+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:49.343804+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10585,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":377},{"shards":"2","objects":3118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12207","reported_seq":12569,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.344040+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:49.344040+0000","last_peered":"2024-09-15T21:23:49.344040+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:49.344040+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":420},{"shards":"2","objects":3110}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12368","reported_seq":13036,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.739692+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:48.739692+0000","last_peered":"2024-09-15T21:23:48.739692+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:48.739692+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3175},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12151","reported_seq":13650,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.720324+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:48.720324+0000","last_peered":"2024-09-15T21:23:48.720324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:48.720324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9736,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3559},{"shards":"0,2","objects":313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12149","reported_seq":12759,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.711278+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:48.711278+0000","last_peered":"2024-09-15T21:23:48.711278+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:48.711278+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25513984,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9796,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3103},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12360","reported_seq":19453,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.759272+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:48.759272+0000","last_peered":"2024-09-15T21:23:48.759272+0000","last_clean":"2024-09-15T21:23:48.759272+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:48.759272+0000","last_undegraded":"2024-09-15T21:23:48.759272+0000","last_fullsized":"2024-09-15T21:23:48.759272+0000","mapping_epoch":47,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633385,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":411800193,"num_objects":100539,"num_object_clones":0,"num_object_copies":201078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100539,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167240,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12787,"num_bytes_recovered":43290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974843904,"data_stored":974780096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554760,"kb_used_data":310648,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93817080,"statfs":{"total":96636764160,"available":96068689920,"internally_reserved":0,"allocated":318103552,"data_stored":317812695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712344,"kb_used_data":355848,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93659496,"statfs":{"total":96636764160,"available":95907323904,"internally_reserved":0,"allocated":364388352,"data_stored":364101567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,4,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590376,"kb_used_data":287956,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93781464,"statfs":{"total":96636764160,"available":96032219136,"internally_reserved":0,"allocated":294866944,"data_stored":294530860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362975849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294252544,"data_stored":294199766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317612032,"data_stored":317604481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:51.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:51.658 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:51.699 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-15T21:23:50.448499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":411800212,"num_objects":100541,"num_object_clones":0,"num_object_copies":201082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167242,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12788,"num_bytes_recovered":43290643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857480,"kb_used_data":954452,"kb_used_omap":12,"kb_used_meta":902963,"kb_avail":281258040,"statfs":{"total":289910292480,"available":288008232960,"internally_reserved":0,"allocated":977358848,"data_stored":976445122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":924634122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,7,9,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25288704,"num_objects":-6174,"num_object_clones":0,"num_object_copies":-12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":-496,"num_objects_unfound":0,"num_objects_dirty":-6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":746,"num_bytes_recovered":2031616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009259"},"pg_stats":[{"pgid":"2.f","version":"50'12213","reported_seq":12874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.701710+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:48.701710+0000","last_peered":"2024-09-15T21:23:48.701710+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:48.701710+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25800704,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12331","reported_seq":12436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.403812+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:49.403812+0000","last_peered":"2024-09-15T21:23:49.403812+0000","last_clean":"2024-09-15T21:23:49.403812+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:49.403812+0000","last_undegraded":"2024-09-15T21:23:49.403812+0000","last_fullsized":"2024-09-15T21:23:49.403812+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12417","reported_seq":19342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.634949+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:48.634949+0000","last_peered":"2024-09-15T21:23:48.634949+0000","last_clean":"2024-09-15T21:23:48.634949+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:48.634949+0000","last_undegraded":"2024-09-15T21:23:48.634949+0000","last_fullsized":"2024-09-15T21:23:48.634949+0000","mapping_epoch":45,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26427416,"num_objects":6453,"num_object_clones":0,"num_object_copies":12906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11140,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12334","reported_seq":12412,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.377994+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:49.377994+0000","last_peered":"2024-09-15T21:23:49.377994+0000","last_clean":"2024-09-15T21:23:49.377994+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:49.377994+0000","last_undegraded":"2024-09-15T21:23:49.377994+0000","last_fullsized":"2024-09-15T21:23:49.377994+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12349","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.630149+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:48.630149+0000","last_peered":"2024-09-15T21:23:48.630149+0000","last_clean":"2024-09-15T21:23:48.630149+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:48.630149+0000","last_undegraded":"2024-09-15T21:23:48.630149+0000","last_fullsized":"2024-09-15T21:23:48.630149+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26107904,"num_objects":6374,"num_object_clones":0,"num_object_copies":12748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10207,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12157","reported_seq":12910,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.703568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:48.703568+0000","last_peered":"2024-09-15T21:23:48.703568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:48.703568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12496","reported_seq":13154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.600563+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:48.600563+0000","last_peered":"2024-09-15T21:23:48.600563+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:48.600563+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3200},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12216","reported_seq":12344,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.696337+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:48.696337+0000","last_peered":"2024-09-15T21:23:48.696337+0000","last_clean":"2024-09-15T21:23:48.696337+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:48.696337+0000","last_undegraded":"2024-09-15T21:23:48.696337+0000","last_fullsized":"2024-09-15T21:23:48.696337+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10001,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12115","reported_seq":18938,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.758256+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:48.758256+0000","last_peered":"2024-09-15T21:23:48.758256+0000","last_clean":"2024-09-15T21:23:48.758256+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:48.758256+0000","last_undegraded":"2024-09-15T21:23:48.758256+0000","last_fullsized":"2024-09-15T21:23:48.758256+0000","mapping_epoch":49,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10701,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11929","reported_seq":12631,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.666935+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:48.666935+0000","last_peered":"2024-09-15T21:23:48.666935+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:48.666935+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9455,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12034","reported_seq":12418,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.343804+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:49.343804+0000","last_peered":"2024-09-15T21:23:49.343804+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:49.343804+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10585,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":377},{"shards":"2","objects":3118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12207","reported_seq":12569,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.344040+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:49.344040+0000","last_peered":"2024-09-15T21:23:49.344040+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:49.344040+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":420},{"shards":"2","objects":3110}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12368","reported_seq":13036,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.739692+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:48.739692+0000","last_peered":"2024-09-15T21:23:48.739692+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:48.739692+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3175},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12151","reported_seq":13650,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.720324+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:48.720324+0000","last_peered":"2024-09-15T21:23:48.720324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:48.720324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9736,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3559},{"shards":"0,2","objects":313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12149","reported_seq":12759,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.711278+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:48.711278+0000","last_peered":"2024-09-15T21:23:48.711278+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:48.711278+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25513984,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9796,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3103},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12360","reported_seq":19453,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.759272+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:48.759272+0000","last_peered":"2024-09-15T21:23:48.759272+0000","last_clean":"2024-09-15T21:23:48.759272+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:48.759272+0000","last_undegraded":"2024-09-15T21:23:48.759272+0000","last_fullsized":"2024-09-15T21:23:48.759272+0000","mapping_epoch":47,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633385,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":411800193,"num_objects":100539,"num_object_clones":0,"num_object_copies":201078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100539,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167240,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12787,"num_bytes_recovered":43290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974843904,"data_stored":974780096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554760,"kb_used_data":310648,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93817080,"statfs":{"total":96636764160,"available":96068689920,"internally_reserved":0,"allocated":318103552,"data_stored":317812695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712344,"kb_used_data":355848,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93659496,"statfs":{"total":96636764160,"available":95907323904,"internally_reserved":0,"allocated":364388352,"data_stored":364101567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,4,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590376,"kb_used_data":287956,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93781464,"statfs":{"total":96636764160,"available":96032219136,"internally_reserved":0,"allocated":294866944,"data_stored":294530860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362975849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294252544,"data_stored":294199766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317612032,"data_stored":317604481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12213", 'reported_seq': 12874, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.701710+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:48.701710+0000', 'last_peered': '2024-09-15T21:23:48.701710+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:48.701710+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25800704, 'num_objects': 6299, 'num_object_clones': 0, 'num_object_copies': 12598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12213, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 288, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3220}, {'shards': '0,2', 'objects': 410}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12157", 'reported_seq': 12910, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.703568+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:48.703568+0000', 'last_peered': '2024-09-15T21:23:48.703568+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:48.703568+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25493504, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9582, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 308, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3161}, {'shards': '0,2', 'objects': 407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:51.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'12496", 'reported_seq': 13154, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.600563+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:48.600563+0000', 'last_peered': '2024-09-15T21:23:48.600563+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:48.600563+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26247168, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3210, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10132, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3200}, {'shards': '0,2', 'objects': 428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'11929", 'reported_seq': 12631, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.666935+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:48.666935+0000', 'last_peered': '2024-09-15T21:23:48.666935+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:48.666935+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'1900", 'ondisk_log_start': "37'1900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25006080, 'num_objects': 6105, 'num_object_clones': 0, 'num_object_copies': 12210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9455, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 285, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3049}, {'shards': '0,2', 'objects': 433}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12034", 'reported_seq': 12418, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:49.343804+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:49.343804+0000', 'last_peered': '2024-09-15T21:23:49.343804+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:49.343804+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25604096, 'num_objects': 6251, 'num_object_clones': 0, 'num_object_copies': 12502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10585, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 133, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 377}, {'shards': '2', 'objects': 3118}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'12207", 'reported_seq': 12569, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:49.344040+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:49.344040+0000', 'last_peered': '2024-09-15T21:23:49.344040+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:49.344040+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25972736, 'num_objects': 6341, 'num_object_clones': 0, 'num_object_copies': 12682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3233, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10606, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 119, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 420}, {'shards': '2', 'objects': 3110}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'12368", 'reported_seq': 13036, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.739692+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:48.739692+0000', 'last_peered': '2024-09-15T21:23:48.739692+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:48.739692+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25976832, 'num_objects': 6342, 'num_object_clones': 0, 'num_object_copies': 12684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3168, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9675, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 266, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3175}, {'shards': '0,2', 'objects': 421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:51.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12151", 'reported_seq': 13650, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.720324+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:48.720324+0000', 'last_peered': '2024-09-15T21:23:48.720324+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:48.720324+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25645056, 'num_objects': 6261, 'num_object_clones': 0, 'num_object_copies': 12522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2389, 'num_objects_misplaced': 313, 'num_objects_unfound': 0, 'num_objects_dirty': 6261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9736, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 680, 'num_bytes_recovered': 2039808, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3559}, {'shards': '0,2', 'objects': 313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:51.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12149", 'reported_seq': 12759, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.711278+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:48.711278+0000', 'last_peered': '2024-09-15T21:23:48.711278+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:48.711278+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25513984, 'num_objects': 6229, 'num_object_clones': 0, 'num_object_copies': 12458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9796, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 239, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3103}, {'shards': '0,2', 'objects': 407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:52.391 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:52.391 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27465/201082 objects degraded (13.659%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100541,"data_bytes":411800212,"bytes_used":1902059520,"bytes_avail":288008232960,"bytes_total":289910292480,"degraded_objects":27465,"degraded_total":201082,"degraded_ratio":0.13658606936473677,"misplaced_objects":313,"misplaced_total":201082,"misplaced_ratio":0.0015565789081071404,"recovering_objects_per_sec":62,"recovering_bytes_per_sec":169170,"recovering_keys_per_sec":0,"num_objects_recovered":746,"num_bytes_recovered":2031616,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":514},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:52.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:52.460 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:53.330 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:53.331 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:53.381 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-15T21:23:52.448987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":411800212,"num_objects":100541,"num_object_clones":0,"num_object_copies":201082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167242,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12788,"num_bytes_recovered":43290643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857480,"kb_used_data":954452,"kb_used_omap":12,"kb_used_meta":902963,"kb_avail":281258040,"statfs":{"total":289910292480,"available":288008232960,"internally_reserved":0,"allocated":977358848,"data_stored":976445122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":924634122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,7,9,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16678912,"num_objects":-4072,"num_object_clones":0,"num_object_copies":-8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":-4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009329"},"pg_stats":[{"pgid":"2.f","version":"50'12213","reported_seq":12874,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.701710+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:48.701710+0000","last_peered":"2024-09-15T21:23:48.701710+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:48.701710+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25800704,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3220},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12331","reported_seq":12436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.403812+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:49.403812+0000","last_peered":"2024-09-15T21:23:49.403812+0000","last_clean":"2024-09-15T21:23:49.403812+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:49.403812+0000","last_undegraded":"2024-09-15T21:23:49.403812+0000","last_fullsized":"2024-09-15T21:23:49.403812+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12417","reported_seq":19342,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.634949+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:48.634949+0000","last_peered":"2024-09-15T21:23:48.634949+0000","last_clean":"2024-09-15T21:23:48.634949+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:48.634949+0000","last_undegraded":"2024-09-15T21:23:48.634949+0000","last_fullsized":"2024-09-15T21:23:48.634949+0000","mapping_epoch":45,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26427416,"num_objects":6453,"num_object_clones":0,"num_object_copies":12906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6453,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11140,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12334","reported_seq":12412,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:49.377994+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:49.377994+0000","last_peered":"2024-09-15T21:23:49.377994+0000","last_clean":"2024-09-15T21:23:49.377994+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:49.377994+0000","last_undegraded":"2024-09-15T21:23:49.377994+0000","last_fullsized":"2024-09-15T21:23:49.377994+0000","mapping_epoch":36,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25542656,"num_objects":6236,"num_object_clones":0,"num_object_copies":12472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12334,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12349","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.630149+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:48.630149+0000","last_peered":"2024-09-15T21:23:48.630149+0000","last_clean":"2024-09-15T21:23:48.630149+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:48.630149+0000","last_undegraded":"2024-09-15T21:23:48.630149+0000","last_fullsized":"2024-09-15T21:23:48.630149+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26107904,"num_objects":6374,"num_object_clones":0,"num_object_copies":12748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10207,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12157","reported_seq":12910,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.703568+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:48.703568+0000","last_peered":"2024-09-15T21:23:48.703568+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:48.703568+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3161},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12496","reported_seq":13154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.600563+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:48.600563+0000","last_peered":"2024-09-15T21:23:48.600563+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:48.600563+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26247168,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3200},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12216","reported_seq":12344,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.696337+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:48.696337+0000","last_peered":"2024-09-15T21:23:48.696337+0000","last_clean":"2024-09-15T21:23:48.696337+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:48.696337+0000","last_undegraded":"2024-09-15T21:23:48.696337+0000","last_fullsized":"2024-09-15T21:23:48.696337+0000","mapping_epoch":35,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10001,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12115","reported_seq":18938,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.758256+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:48.758256+0000","last_peered":"2024-09-15T21:23:48.758256+0000","last_clean":"2024-09-15T21:23:48.758256+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:48.758256+0000","last_undegraded":"2024-09-15T21:23:48.758256+0000","last_fullsized":"2024-09-15T21:23:48.758256+0000","mapping_epoch":49,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10701,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658267+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658267+0000","last_peered":"2024-09-15T21:23:34.658267+0000","last_clean":"2024-09-15T21:23:34.658267+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658267+0000","last_undegraded":"2024-09-15T21:23:34.658267+0000","last_fullsized":"2024-09-15T21:23:34.658267+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'11929","reported_seq":12631,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.666935+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:48.666935+0000","last_peered":"2024-09-15T21:23:48.666935+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:48.666935+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'1900","ondisk_log_start":"37'1900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9455,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12034","reported_seq":12418,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.343804+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:49.343804+0000","last_peered":"2024-09-15T21:23:49.343804+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:49.343804+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10585,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":377},{"shards":"2","objects":3118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12207","reported_seq":12569,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:49.344040+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:49.344040+0000","last_peered":"2024-09-15T21:23:49.344040+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:49.344040+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":420},{"shards":"2","objects":3110}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12368","reported_seq":13036,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.739692+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:48.739692+0000","last_peered":"2024-09-15T21:23:48.739692+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:48.739692+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9675,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3175},{"shards":"0,2","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12151","reported_seq":13650,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.720324+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:48.720324+0000","last_peered":"2024-09-15T21:23:48.720324+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:48.720324+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9736,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":680,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3559},{"shards":"0,2","objects":313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665494+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665494+0000","last_peered":"2024-09-15T21:23:34.665494+0000","last_clean":"2024-09-15T21:23:34.665494+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665494+0000","last_undegraded":"2024-09-15T21:23:34.665494+0000","last_fullsized":"2024-09-15T21:23:34.665494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12149","reported_seq":12759,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.711278+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:48.711278+0000","last_peered":"2024-09-15T21:23:48.711278+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:48.711278+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25513984,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9796,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3103},{"shards":"0,2","objects":407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12360","reported_seq":19453,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.759272+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:48.759272+0000","last_peered":"2024-09-15T21:23:48.759272+0000","last_clean":"2024-09-15T21:23:48.759272+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:48.759272+0000","last_undegraded":"2024-09-15T21:23:48.759272+0000","last_fullsized":"2024-09-15T21:23:48.759272+0000","mapping_epoch":47,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633385,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":411800193,"num_objects":100539,"num_object_clones":0,"num_object_copies":201078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27465,"num_objects_misplaced":313,"num_objects_unfound":0,"num_objects_dirty":100539,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167240,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12787,"num_bytes_recovered":43290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974843904,"data_stored":974780096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953551,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554760,"kb_used_data":310648,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93817080,"statfs":{"total":96636764160,"available":96068689920,"internally_reserved":0,"allocated":318103552,"data_stored":317812695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712344,"kb_used_data":355848,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93659496,"statfs":{"total":96636764160,"available":95907323904,"internally_reserved":0,"allocated":364388352,"data_stored":364101567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,4,0,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590376,"kb_used_data":287956,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93781464,"statfs":{"total":96636764160,"available":96032219136,"internally_reserved":0,"allocated":294866944,"data_stored":294530860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":362979328,"data_stored":362975849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294252544,"data_stored":294199766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317612032,"data_stored":317604481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:56.383 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:57.125 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:57.134 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:57.195 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-15T21:23:56.449903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403059348,"num_objects":98407,"num_object_clones":0,"num_object_copies":196814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169376,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13033,"num_bytes_recovered":43974675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880188,"kb_used_data":942476,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281235332,"statfs":{"total":289910292480,"available":287984979968,"internally_reserved":0,"allocated":965095424,"data_stored":964181698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":960154634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,8,3,4,3,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19169280,"num_objects":-4680,"num_object_clones":0,"num_object_copies":-9360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":-4680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4680,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006774"},"pg_stats":[{"pgid":"2.f","version":"50'12327","reported_seq":13014,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.766199+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:53.766199+0000","last_peered":"2024-09-15T21:23:53.766199+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:53.766199+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12472","reported_seq":12577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.589660+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:54.589660+0000","last_peered":"2024-09-15T21:23:54.589660+0000","last_clean":"2024-09-15T21:23:54.589660+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:54.589660+0000","last_undegraded":"2024-09-15T21:23:54.589660+0000","last_fullsized":"2024-09-15T21:23:54.589660+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11198,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12552","reported_seq":19477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.750752+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:53.750752+0000","last_peered":"2024-09-15T21:23:53.750752+0000","last_clean":"2024-09-15T21:23:53.750752+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:53.750752+0000","last_undegraded":"2024-09-15T21:23:53.750752+0000","last_fullsized":"2024-09-15T21:23:53.750752+0000","mapping_epoch":45,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25874456,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11275,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12477","reported_seq":12555,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.585064+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:54.585064+0000","last_peered":"2024-09-15T21:23:54.585064+0000","last_clean":"2024-09-15T21:23:54.585064+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:54.585064+0000","last_undegraded":"2024-09-15T21:23:54.585064+0000","last_fullsized":"2024-09-15T21:23:54.585064+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12477,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12475","reported_seq":12602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.691736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:53.691736+0000","last_peered":"2024-09-15T21:23:53.691736+0000","last_clean":"2024-09-15T21:23:53.691736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:53.691736+0000","last_undegraded":"2024-09-15T21:23:53.691736+0000","last_fullsized":"2024-09-15T21:23:53.691736+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25591808,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10333,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12285","reported_seq":13053,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.757149+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:53.757149+0000","last_peered":"2024-09-15T21:23:53.757149+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:53.757149+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9710,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12635","reported_seq":13309,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.693940+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:53.693940+0000","last_peered":"2024-09-15T21:23:53.693940+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:53.693940+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3069},{"shards":"0,2","objects":289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12347","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.763634+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:53.763634+0000","last_peered":"2024-09-15T21:23:53.763634+0000","last_clean":"2024-09-15T21:23:53.763634+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:53.763634+0000","last_undegraded":"2024-09-15T21:23:53.763634+0000","last_fullsized":"2024-09-15T21:23:53.763634+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809472,"num_objects":6057,"num_object_clones":0,"num_object_copies":12114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12226","reported_seq":19049,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.833886+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:53.833886+0000","last_peered":"2024-09-15T21:23:53.833886+0000","last_clean":"2024-09-15T21:23:53.833886+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:53.833886+0000","last_undegraded":"2024-09-15T21:23:53.833886+0000","last_fullsized":"2024-09-15T21:23:53.833886+0000","mapping_epoch":49,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12072","reported_seq":12796,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.744692+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:53.744692+0000","last_peered":"2024-09-15T21:23:53.744692+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:53.744692+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2919},{"shards":"0,2","objects":290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12180","reported_seq":12594,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.572647+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:54.572647+0000","last_peered":"2024-09-15T21:23:54.572647+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:54.572647+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10731,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":231},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12357","reported_seq":12735,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.593019+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:54.593019+0000","last_peered":"2024-09-15T21:23:54.593019+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:54.593019+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25358336,"num_objects":6191,"num_object_clones":0,"num_object_copies":12382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":270},{"shards":"2","objects":2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12501","reported_seq":13192,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:53.755948+0000","last_peered":"2024-09-15T21:23:53.755948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:53.755948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9808,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12276","reported_seq":14109,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755587+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:53.755587+0000","last_peered":"2024-09-15T21:23:53.755587+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:53.755587+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9861,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":2723840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12269","reported_seq":12886,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.640758+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:53.640758+0000","last_peered":"2024-09-15T21:23:53.640758+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:53.640758+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12509","reported_seq":19602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.887553+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:53.887553+0000","last_peered":"2024-09-15T21:23:53.887553+0000","last_clean":"2024-09-15T21:23:53.887553+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:53.887553+0000","last_undegraded":"2024-09-15T21:23:53.887553+0000","last_fullsized":"2024-09-15T21:23:53.887553+0000","mapping_epoch":47,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25023081,"num_objects":6110,"num_object_clones":0,"num_object_copies":12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403059329,"num_objects":98405,"num_object_clones":0,"num_object_copies":196810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169374,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13032,"num_bytes_recovered":43974656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962383872,"data_stored":962320064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953552,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572760,"kb_used_data":306440,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799080,"statfs":{"total":96636764160,"available":96050257920,"internally_reserved":0,"allocated":313794560,"data_stored":313503703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710132,"kb_used_data":349476,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93661708,"statfs":{"total":96636764160,"available":95909588992,"internally_reserved":0,"allocated":357863424,"data_stored":357576639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":369290929},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,1,1,2,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597296,"kb_used_data":286560,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93774544,"statfs":{"total":96636764160,"available":96025133056,"internally_reserved":0,"allocated":293437440,"data_stored":293101356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356511744,"data_stored":356508265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292757504,"data_stored":292704726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313114624,"data_stored":313107073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:57.196 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:57.914 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:57.915 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:57.938 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-15T21:23:56.449903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403059348,"num_objects":98407,"num_object_clones":0,"num_object_copies":196814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169376,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13033,"num_bytes_recovered":43974675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880188,"kb_used_data":942476,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281235332,"statfs":{"total":289910292480,"available":287984979968,"internally_reserved":0,"allocated":965095424,"data_stored":964181698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":960154634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,8,3,4,3,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19169280,"num_objects":-4680,"num_object_clones":0,"num_object_copies":-9360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-179,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":-4680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4680,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006774"},"pg_stats":[{"pgid":"2.f","version":"50'12327","reported_seq":13014,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.766199+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:53.766199+0000","last_peered":"2024-09-15T21:23:53.766199+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:53.766199+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12472","reported_seq":12577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.589660+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:54.589660+0000","last_peered":"2024-09-15T21:23:54.589660+0000","last_clean":"2024-09-15T21:23:54.589660+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:54.589660+0000","last_undegraded":"2024-09-15T21:23:54.589660+0000","last_fullsized":"2024-09-15T21:23:54.589660+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11198,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12552","reported_seq":19477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.750752+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:53.750752+0000","last_peered":"2024-09-15T21:23:53.750752+0000","last_clean":"2024-09-15T21:23:53.750752+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:53.750752+0000","last_undegraded":"2024-09-15T21:23:53.750752+0000","last_fullsized":"2024-09-15T21:23:53.750752+0000","mapping_epoch":45,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25874456,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11275,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12477","reported_seq":12555,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.585064+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:54.585064+0000","last_peered":"2024-09-15T21:23:54.585064+0000","last_clean":"2024-09-15T21:23:54.585064+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:54.585064+0000","last_undegraded":"2024-09-15T21:23:54.585064+0000","last_fullsized":"2024-09-15T21:23:54.585064+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12477,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12475","reported_seq":12602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.691736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:53.691736+0000","last_peered":"2024-09-15T21:23:53.691736+0000","last_clean":"2024-09-15T21:23:53.691736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:53.691736+0000","last_undegraded":"2024-09-15T21:23:53.691736+0000","last_fullsized":"2024-09-15T21:23:53.691736+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25591808,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10333,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12285","reported_seq":13053,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.757149+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:53.757149+0000","last_peered":"2024-09-15T21:23:53.757149+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:53.757149+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9710,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12635","reported_seq":13309,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.693940+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:53.693940+0000","last_peered":"2024-09-15T21:23:53.693940+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:53.693940+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3069},{"shards":"0,2","objects":289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12347","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.763634+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:53.763634+0000","last_peered":"2024-09-15T21:23:53.763634+0000","last_clean":"2024-09-15T21:23:53.763634+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:53.763634+0000","last_undegraded":"2024-09-15T21:23:53.763634+0000","last_fullsized":"2024-09-15T21:23:53.763634+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809472,"num_objects":6057,"num_object_clones":0,"num_object_copies":12114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12226","reported_seq":19049,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.833886+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:53.833886+0000","last_peered":"2024-09-15T21:23:53.833886+0000","last_clean":"2024-09-15T21:23:53.833886+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:53.833886+0000","last_undegraded":"2024-09-15T21:23:53.833886+0000","last_fullsized":"2024-09-15T21:23:53.833886+0000","mapping_epoch":49,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12072","reported_seq":12796,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.744692+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:53.744692+0000","last_peered":"2024-09-15T21:23:53.744692+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:53.744692+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2919},{"shards":"0,2","objects":290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12180","reported_seq":12594,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.572647+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:54.572647+0000","last_peered":"2024-09-15T21:23:54.572647+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:54.572647+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10731,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":231},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12357","reported_seq":12735,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.593019+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:54.593019+0000","last_peered":"2024-09-15T21:23:54.593019+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:54.593019+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25358336,"num_objects":6191,"num_object_clones":0,"num_object_copies":12382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":270},{"shards":"2","objects":2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12501","reported_seq":13192,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:53.755948+0000","last_peered":"2024-09-15T21:23:53.755948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:53.755948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9808,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12276","reported_seq":14109,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755587+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:53.755587+0000","last_peered":"2024-09-15T21:23:53.755587+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:53.755587+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9861,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":2723840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12269","reported_seq":12886,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.640758+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:53.640758+0000","last_peered":"2024-09-15T21:23:53.640758+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:53.640758+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12509","reported_seq":19602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.887553+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:53.887553+0000","last_peered":"2024-09-15T21:23:53.887553+0000","last_clean":"2024-09-15T21:23:53.887553+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:53.887553+0000","last_undegraded":"2024-09-15T21:23:53.887553+0000","last_fullsized":"2024-09-15T21:23:53.887553+0000","mapping_epoch":47,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25023081,"num_objects":6110,"num_object_clones":0,"num_object_copies":12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403059329,"num_objects":98405,"num_object_clones":0,"num_object_copies":196810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169374,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13032,"num_bytes_recovered":43974656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962383872,"data_stored":962320064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953552,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572760,"kb_used_data":306440,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799080,"statfs":{"total":96636764160,"available":96050257920,"internally_reserved":0,"allocated":313794560,"data_stored":313503703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710132,"kb_used_data":349476,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93661708,"statfs":{"total":96636764160,"available":95909588992,"internally_reserved":0,"allocated":357863424,"data_stored":357576639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":369290929},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,1,1,2,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597296,"kb_used_data":286560,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93774544,"statfs":{"total":96636764160,"available":96025133056,"internally_reserved":0,"allocated":293437440,"data_stored":293101356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356511744,"data_stored":356508265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292757504,"data_stored":292704726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313114624,"data_stored":313107073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12327", 'reported_seq': 13014, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.766199+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:53.766199+0000', 'last_peered': '2024-09-15T21:23:53.766199+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:53.766199+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25333760, 'num_objects': 6185, 'num_object_clones': 0, 'num_object_copies': 12370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3066, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12327, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 301, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3119}, {'shards': '0,2', 'objects': 296}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12285", 'reported_seq': 13053, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.757149+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:53.757149+0000', 'last_peered': '2024-09-15T21:23:53.757149+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:53.757149+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24969216, 'num_objects': 6096, 'num_object_clones': 0, 'num_object_copies': 12192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9710, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 315, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3040}, {'shards': '0,2', 'objects': 279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'12635", 'reported_seq': 13309, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.693940+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:53.693940+0000', 'last_peered': '2024-09-15T21:23:53.693940+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:53.693940+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25677824, 'num_objects': 6269, 'num_object_clones': 0, 'num_object_copies': 12538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3202, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10271, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 271, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3069}, {'shards': '0,2', 'objects': 289}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:57.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12072", 'reported_seq': 12796, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.744692+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:53.744692+0000', 'last_peered': '2024-09-15T21:23:53.744692+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:53.744692+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'2000", 'ondisk_log_start': "37'2000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24420352, 'num_objects': 5962, 'num_object_clones': 0, 'num_object_copies': 11924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3045, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9598, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 298, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2919}, {'shards': '0,2', 'objects': 290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12180", 'reported_seq': 12594, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:54.572647+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:54.572647+0000', 'last_peered': '2024-09-15T21:23:54.572647+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:54.572647+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'2100", 'ondisk_log_start': "37'2100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25006080, 'num_objects': 6105, 'num_object_clones': 0, 'num_object_copies': 12210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10731, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 148, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 231}, {'shards': '2', 'objects': 2987}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'12357", 'reported_seq': 12735, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:54.593019+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:54.593019+0000', 'last_peered': '2024-09-15T21:23:54.593019+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:54.593019+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25358336, 'num_objects': 6191, 'num_object_clones': 0, 'num_object_copies': 12382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3225, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10756, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 127, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 270}, {'shards': '2', 'objects': 2968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'12501", 'reported_seq': 13192, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.755948+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:53.755948+0000', 'last_peered': '2024-09-15T21:23:53.755948+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:53.755948+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25432064, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9808, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 277, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3053}, {'shards': '0,2', 'objects': 288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12276", 'reported_seq': 14109, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.755587+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:53.755587+0000', 'last_peered': '2024-09-15T21:23:53.755587+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:53.755587+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25133056, 'num_objects': 6136, 'num_object_clones': 0, 'num_object_copies': 12272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2389, 'num_objects_misplaced': 146, 'num_objects_unfound': 0, 'num_objects_dirty': 6136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9861, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 847, 'num_bytes_recovered': 2723840, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3601}, {'shards': '0,2', 'objects': 146}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:57.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12269", 'reported_seq': 12886, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.640758+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:53.640758+0000', 'last_peered': '2024-09-15T21:23:53.640758+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:53.640758+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25022464, 'num_objects': 6109, 'num_object_clones': 0, 'num_object_copies': 12218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3125, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9916, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 242, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2986}, {'shards': '0,2', 'objects': 287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:57.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:58.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:58.629 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27387/196814 objects degraded (13.915%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98407,"data_bytes":403059348,"bytes_used":1925312512,"bytes_avail":287984979968,"bytes_total":289910292480,"degraded_objects":27387,"degraded_total":196814,"degraded_ratio":0.13915168636377492,"misplaced_objects":146,"misplaced_total":196814,"misplaced_ratio":0.00074181714715416578,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":113600,"recovering_keys_per_sec":0,"num_objects_recovered":501,"num_bytes_recovered":1363968,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":344},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:58.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:59.443 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:23:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:23:59.558 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-15T21:23:58.450346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403059348,"num_objects":98407,"num_object_clones":0,"num_object_copies":196814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169376,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13033,"num_bytes_recovered":43974675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1880188,"kb_used_data":942476,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281235332,"statfs":{"total":289910292480,"available":287984979968,"internally_reserved":0,"allocated":965095424,"data_stored":964181698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":960154634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,8,3,4,3,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16953344,"num_objects":-4139,"num_object_clones":0,"num_object_copies":-8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":-4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4139,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006705"},"pg_stats":[{"pgid":"2.f","version":"50'12327","reported_seq":13014,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.766199+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:53.766199+0000","last_peered":"2024-09-15T21:23:53.766199+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:53.766199+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12327,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3119},{"shards":"0,2","objects":296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12472","reported_seq":12577,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.589660+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:54.589660+0000","last_peered":"2024-09-15T21:23:54.589660+0000","last_clean":"2024-09-15T21:23:54.589660+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:54.589660+0000","last_undegraded":"2024-09-15T21:23:54.589660+0000","last_fullsized":"2024-09-15T21:23:54.589660+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11198,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12552","reported_seq":19477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.750752+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:53.750752+0000","last_peered":"2024-09-15T21:23:53.750752+0000","last_clean":"2024-09-15T21:23:53.750752+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:53.750752+0000","last_undegraded":"2024-09-15T21:23:53.750752+0000","last_fullsized":"2024-09-15T21:23:53.750752+0000","mapping_epoch":45,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25874456,"num_objects":6318,"num_object_clones":0,"num_object_copies":12636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6318,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11275,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12477","reported_seq":12555,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:54.585064+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:54.585064+0000","last_peered":"2024-09-15T21:23:54.585064+0000","last_clean":"2024-09-15T21:23:54.585064+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:54.585064+0000","last_undegraded":"2024-09-15T21:23:54.585064+0000","last_fullsized":"2024-09-15T21:23:54.585064+0000","mapping_epoch":36,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12477,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12475","reported_seq":12602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.691736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:53.691736+0000","last_peered":"2024-09-15T21:23:53.691736+0000","last_clean":"2024-09-15T21:23:53.691736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:53.691736+0000","last_undegraded":"2024-09-15T21:23:53.691736+0000","last_fullsized":"2024-09-15T21:23:53.691736+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25591808,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10333,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12285","reported_seq":13053,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.757149+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:53.757149+0000","last_peered":"2024-09-15T21:23:53.757149+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:53.757149+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9710,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12635","reported_seq":13309,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.693940+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:53.693940+0000","last_peered":"2024-09-15T21:23:53.693940+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:53.693940+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3069},{"shards":"0,2","objects":289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12347","reported_seq":12476,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.763634+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:53.763634+0000","last_peered":"2024-09-15T21:23:53.763634+0000","last_clean":"2024-09-15T21:23:53.763634+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:53.763634+0000","last_undegraded":"2024-09-15T21:23:53.763634+0000","last_fullsized":"2024-09-15T21:23:53.763634+0000","mapping_epoch":35,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809472,"num_objects":6057,"num_object_clones":0,"num_object_copies":12114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10132,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12226","reported_seq":19049,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.833886+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:53.833886+0000","last_peered":"2024-09-15T21:23:53.833886+0000","last_clean":"2024-09-15T21:23:53.833886+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:53.833886+0000","last_undegraded":"2024-09-15T21:23:53.833886+0000","last_fullsized":"2024-09-15T21:23:53.833886+0000","mapping_epoch":49,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12072","reported_seq":12796,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.744692+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:53.744692+0000","last_peered":"2024-09-15T21:23:53.744692+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:53.744692+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2000","ondisk_log_start":"37'2000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3045,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2919},{"shards":"0,2","objects":290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12180","reported_seq":12594,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.572647+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:54.572647+0000","last_peered":"2024-09-15T21:23:54.572647+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:54.572647+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2100","ondisk_log_start":"37'2100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25006080,"num_objects":6105,"num_object_clones":0,"num_object_copies":12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10731,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":231},{"shards":"2","objects":2987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12357","reported_seq":12735,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:54.593019+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:54.593019+0000","last_peered":"2024-09-15T21:23:54.593019+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:54.593019+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25358336,"num_objects":6191,"num_object_clones":0,"num_object_copies":12382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":270},{"shards":"2","objects":2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12501","reported_seq":13192,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755948+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:53.755948+0000","last_peered":"2024-09-15T21:23:53.755948+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:53.755948+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9808,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3053},{"shards":"0,2","objects":288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12276","reported_seq":14109,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.755587+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:53.755587+0000","last_peered":"2024-09-15T21:23:53.755587+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:53.755587+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25133056,"num_objects":6136,"num_object_clones":0,"num_object_copies":12272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2389,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":6136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9861,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":2723840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12269","reported_seq":12886,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.640758+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:53.640758+0000","last_peered":"2024-09-15T21:23:53.640758+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:53.640758+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12509","reported_seq":19602,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.887553+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:53.887553+0000","last_peered":"2024-09-15T21:23:53.887553+0000","last_clean":"2024-09-15T21:23:53.887553+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:53.887553+0000","last_undegraded":"2024-09-15T21:23:53.887553+0000","last_fullsized":"2024-09-15T21:23:53.887553+0000","mapping_epoch":47,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25023081,"num_objects":6110,"num_object_clones":0,"num_object_copies":12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10169,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403059329,"num_objects":98405,"num_object_clones":0,"num_object_copies":196810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27387,"num_objects_misplaced":146,"num_objects_unfound":0,"num_objects_dirty":98405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169374,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13032,"num_bytes_recovered":43974656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":962383872,"data_stored":962320064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953552,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572760,"kb_used_data":306440,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93799080,"statfs":{"total":96636764160,"available":96050257920,"internally_reserved":0,"allocated":313794560,"data_stored":313503703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149780,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710132,"kb_used_data":349476,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93661708,"statfs":{"total":96636764160,"available":95909588992,"internally_reserved":0,"allocated":357863424,"data_stored":357576639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":369290929},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,1,1,2,2,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597296,"kb_used_data":286560,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93774544,"statfs":{"total":96636764160,"available":96025133056,"internally_reserved":0,"allocated":293437440,"data_stored":293101356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":318172844},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356511744,"data_stored":356508265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292757504,"data_stored":292704726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313114624,"data_stored":313107073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:02.561 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:03.210 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:03.211 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:03.276 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-15T21:24:02.451290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394560148,"num_objects":96332,"num_object_clones":0,"num_object_copies":192664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171451,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13302,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1855464,"kb_used_data":930232,"kb_used_omap":12,"kb_used_meta":925171,"kb_avail":281260056,"statfs":{"total":289910292480,"available":288010297344,"internally_reserved":0,"allocated":952557568,"data_stored":951643842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":947375114},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,6,0,6,9,6,0,11],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17240064,"num_objects":-4209,"num_object_clones":0,"num_object_copies":-8418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":-313,"num_objects_unfound":0,"num_objects_dirty":-4209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4209,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"50'12461","reported_seq":13188,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.747468+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:58.747468+0000","last_peered":"2024-09-15T21:23:58.747468+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:58.747468+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12597","reported_seq":12702,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.596445+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:59.596445+0000","last_peered":"2024-09-15T21:23:59.596445+0000","last_clean":"2024-09-15T21:23:59.596445+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:59.596445+0000","last_undegraded":"2024-09-15T21:23:59.596445+0000","last_fullsized":"2024-09-15T21:23:59.596445+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24752128,"num_objects":6043,"num_object_clones":0,"num_object_copies":12086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11323,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12671","reported_seq":19596,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.864519+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:58.864519+0000","last_peered":"2024-09-15T21:23:58.864519+0000","last_clean":"2024-09-15T21:23:58.864519+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:58.864519+0000","last_undegraded":"2024-09-15T21:23:58.864519+0000","last_fullsized":"2024-09-15T21:23:58.864519+0000","mapping_epoch":45,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387032,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11394,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12607","reported_seq":12685,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.490209+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:59.490209+0000","last_peered":"2024-09-15T21:23:59.490209+0000","last_clean":"2024-09-15T21:23:59.490209+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:59.490209+0000","last_undegraded":"2024-09-15T21:23:59.490209+0000","last_fullsized":"2024-09-15T21:23:59.490209+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12607,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12622","reported_seq":12749,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.706426+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:58.706426+0000","last_peered":"2024-09-15T21:23:58.706426+0000","last_clean":"2024-09-15T21:23:58.706426+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:58.706426+0000","last_undegraded":"2024-09-15T21:23:58.706426+0000","last_fullsized":"2024-09-15T21:23:58.706426+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10480,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12405","reported_seq":13203,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.712480+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:58.712480+0000","last_peered":"2024-09-15T21:23:58.712480+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:58.712480+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9830,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2935},{"shards":"0,2","objects":159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12747","reported_seq":13444,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.713337+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:58.713337+0000","last_peered":"2024-09-15T21:23:58.713337+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:58.713337+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10383,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12470","reported_seq":12599,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.774885+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:58.774885+0000","last_peered":"2024-09-15T21:23:58.774885+0000","last_clean":"2024-09-15T21:23:58.774885+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:58.774885+0000","last_undegraded":"2024-09-15T21:23:58.774885+0000","last_fullsized":"2024-09-15T21:23:58.774885+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24305664,"num_objects":5934,"num_object_clones":0,"num_object_copies":11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12377","reported_seq":19200,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.852655+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:58.852655+0000","last_peered":"2024-09-15T21:23:58.852655+0000","last_clean":"2024-09-15T21:23:58.852655+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:58.852655+0000","last_undegraded":"2024-09-15T21:23:58.852655+0000","last_fullsized":"2024-09-15T21:23:58.852655+0000","mapping_epoch":49,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24567808,"num_objects":5998,"num_object_clones":0,"num_object_copies":11996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12205","reported_seq":12961,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.748489+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:58.748489+0000","last_peered":"2024-09-15T21:23:58.748489+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:58.748489+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23875584,"num_objects":5829,"num_object_clones":0,"num_object_copies":11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2800},{"shards":"0,2","objects":157}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12293","reported_seq":12714,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.580288+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:59.580288+0000","last_peered":"2024-09-15T21:23:59.580288+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:59.580288+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24543232,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10844,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":118},{"shards":"2","objects":2877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12495","reported_seq":12891,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.564439+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:59.564439+0000","last_peered":"2024-09-15T21:23:59.564439+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:59.564439+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":132},{"shards":"2","objects":2839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12617","reported_seq":13332,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.751500+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:58.751500+0000","last_peered":"2024-09-15T21:23:58.751500+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:58.751500+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9924,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2949},{"shards":"0,2","objects":172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12418","reported_seq":14596,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.753358+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:58.753358+0000","last_peered":"2024-09-15T21:23:58.753358+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:58.753358+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24551424,"num_objects":5994,"num_object_clones":0,"num_object_copies":11988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10003,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12401","reported_seq":13040,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.694802+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:58.694802+0000","last_peered":"2024-09-15T21:23:58.694802+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:58.694802+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10048,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12649","reported_seq":19742,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.880849+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:58.880849+0000","last_peered":"2024-09-15T21:23:58.880849+0000","last_clean":"2024-09-15T21:23:58.880849+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:58.880849+0000","last_undegraded":"2024-09-15T21:23:58.880849+0000","last_fullsized":"2024-09-15T21:23:58.880849+0000","mapping_epoch":47,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449641,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394560129,"num_objects":96330,"num_object_clones":0,"num_object_copies":192660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171449,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13301,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950001664,"data_stored":949937856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953553,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572784,"kb_used_data":302304,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93799056,"statfs":{"total":96636764160,"available":96050233344,"internally_reserved":0,"allocated":309559296,"data_stored":309268439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695588,"kb_used_data":343252,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93676252,"statfs":{"total":96636764160,"available":95924482048,"internally_reserved":0,"allocated":351490048,"data_stored":351203263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,2,2,0,10],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587092,"kb_used_data":284676,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93784748,"statfs":{"total":96636764160,"available":96035581952,"internally_reserved":0,"allocated":291508224,"data_stored":291172140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350105600,"data_stored":350102121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291082240,"data_stored":291029462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308813824,"data_stored":308806273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:03.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:03.958 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:04.003 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-15T21:24:02.451290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394560148,"num_objects":96332,"num_object_clones":0,"num_object_copies":192664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96332,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171451,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13302,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1855464,"kb_used_data":930232,"kb_used_omap":12,"kb_used_meta":925171,"kb_avail":281260056,"statfs":{"total":289910292480,"available":288010297344,"internally_reserved":0,"allocated":952557568,"data_stored":951643842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":947375114},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,6,0,6,9,6,0,11],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17240064,"num_objects":-4209,"num_object_clones":0,"num_object_copies":-8418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":-313,"num_objects_unfound":0,"num_objects_dirty":-4209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4209,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":514,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"50'12461","reported_seq":13188,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.747468+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:23:58.747468+0000","last_peered":"2024-09-15T21:23:58.747468+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:23:58.747468+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24784896,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12597","reported_seq":12702,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.596445+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:59.596445+0000","last_peered":"2024-09-15T21:23:59.596445+0000","last_clean":"2024-09-15T21:23:59.596445+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:59.596445+0000","last_undegraded":"2024-09-15T21:23:59.596445+0000","last_fullsized":"2024-09-15T21:23:59.596445+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24752128,"num_objects":6043,"num_object_clones":0,"num_object_copies":12086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11323,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12671","reported_seq":19596,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.864519+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:23:58.864519+0000","last_peered":"2024-09-15T21:23:58.864519+0000","last_clean":"2024-09-15T21:23:58.864519+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:23:58.864519+0000","last_undegraded":"2024-09-15T21:23:58.864519+0000","last_fullsized":"2024-09-15T21:23:58.864519+0000","mapping_epoch":45,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387032,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11394,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12607","reported_seq":12685,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.490209+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:59.490209+0000","last_peered":"2024-09-15T21:23:59.490209+0000","last_clean":"2024-09-15T21:23:59.490209+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:59.490209+0000","last_undegraded":"2024-09-15T21:23:59.490209+0000","last_fullsized":"2024-09-15T21:23:59.490209+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12607,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12622","reported_seq":12749,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.706426+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:23:58.706426+0000","last_peered":"2024-09-15T21:23:58.706426+0000","last_clean":"2024-09-15T21:23:58.706426+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:23:58.706426+0000","last_undegraded":"2024-09-15T21:23:58.706426+0000","last_fullsized":"2024-09-15T21:23:58.706426+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10480,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12405","reported_seq":13203,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.712480+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:23:58.712480+0000","last_peered":"2024-09-15T21:23:58.712480+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:23:58.712480+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9830,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2935},{"shards":"0,2","objects":159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12747","reported_seq":13444,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.713337+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:23:58.713337+0000","last_peered":"2024-09-15T21:23:58.713337+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:23:58.713337+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10383,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12470","reported_seq":12599,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.774885+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:23:58.774885+0000","last_peered":"2024-09-15T21:23:58.774885+0000","last_clean":"2024-09-15T21:23:58.774885+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:23:58.774885+0000","last_undegraded":"2024-09-15T21:23:58.774885+0000","last_fullsized":"2024-09-15T21:23:58.774885+0000","mapping_epoch":35,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24305664,"num_objects":5934,"num_object_clones":0,"num_object_copies":11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12377","reported_seq":19200,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.852655+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:23:58.852655+0000","last_peered":"2024-09-15T21:23:58.852655+0000","last_clean":"2024-09-15T21:23:58.852655+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:23:58.852655+0000","last_undegraded":"2024-09-15T21:23:58.852655+0000","last_fullsized":"2024-09-15T21:23:58.852655+0000","mapping_epoch":49,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24567808,"num_objects":5998,"num_object_clones":0,"num_object_copies":11996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12205","reported_seq":12961,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.748489+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:23:58.748489+0000","last_peered":"2024-09-15T21:23:58.748489+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:23:58.748489+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23875584,"num_objects":5829,"num_object_clones":0,"num_object_copies":11658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2800},{"shards":"0,2","objects":157}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12293","reported_seq":12714,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.580288+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:59.580288+0000","last_peered":"2024-09-15T21:23:59.580288+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:59.580288+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24543232,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10844,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":118},{"shards":"2","objects":2877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12495","reported_seq":12891,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.564439+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:59.564439+0000","last_peered":"2024-09-15T21:23:59.564439+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:59.564439+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":132},{"shards":"2","objects":2839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12617","reported_seq":13332,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.751500+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:23:58.751500+0000","last_peered":"2024-09-15T21:23:58.751500+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:23:58.751500+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9924,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2949},{"shards":"0,2","objects":172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12418","reported_seq":14596,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.753358+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:23:58.753358+0000","last_peered":"2024-09-15T21:23:58.753358+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:23:58.753358+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24551424,"num_objects":5994,"num_object_clones":0,"num_object_copies":11988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10003,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12401","reported_seq":13040,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.694802+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:23:58.694802+0000","last_peered":"2024-09-15T21:23:58.694802+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:23:58.694802+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24481792,"num_objects":5977,"num_object_clones":0,"num_object_copies":11954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10048,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12649","reported_seq":19742,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.880849+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:23:58.880849+0000","last_peered":"2024-09-15T21:23:58.880849+0000","last_clean":"2024-09-15T21:23:58.880849+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:23:58.880849+0000","last_undegraded":"2024-09-15T21:23:58.880849+0000","last_fullsized":"2024-09-15T21:23:58.880849+0000","mapping_epoch":47,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449641,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394560129,"num_objects":96330,"num_object_clones":0,"num_object_copies":192660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171449,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13301,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950001664,"data_stored":949937856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953553,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572784,"kb_used_data":302304,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93799056,"statfs":{"total":96636764160,"available":96050233344,"internally_reserved":0,"allocated":309559296,"data_stored":309268439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695588,"kb_used_data":343252,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93676252,"statfs":{"total":96636764160,"available":95924482048,"internally_reserved":0,"allocated":351490048,"data_stored":351203263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,2,2,0,10],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587092,"kb_used_data":284676,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93784748,"statfs":{"total":96636764160,"available":96035581952,"internally_reserved":0,"allocated":291508224,"data_stored":291172140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":309653164},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":350105600,"data_stored":350102121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":291082240,"data_stored":291029462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308813824,"data_stored":308806273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:04.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:04.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12461", 'reported_seq': 13188, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.747468+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:23:58.747468+0000', 'last_peered': '2024-09-15T21:23:58.747468+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:23:58.747468+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24784896, 'num_objects': 6051, 'num_object_clones': 0, 'num_object_copies': 12102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3046, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12461, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 321, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3005}, {'shards': '0,2', 'objects': 162}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12405", 'reported_seq': 13203, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.712480+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:23:58.712480+0000', 'last_peered': '2024-09-15T21:23:58.712480+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:23:58.712480+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24477696, 'num_objects': 5976, 'num_object_clones': 0, 'num_object_copies': 11952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3043, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9830, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 330, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2935}, {'shards': '0,2', 'objects': 159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'12747", 'reported_seq': 13444, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.713337+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:23:58.713337+0000', 'last_peered': '2024-09-15T21:23:58.713337+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:23:58.713337+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25219072, 'num_objects': 6157, 'num_object_clones': 0, 'num_object_copies': 12314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10383, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 283, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2969}, {'shards': '0,2', 'objects': 177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12205", 'reported_seq': 12961, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.748489+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:23:58.748489+0000', 'last_peered': '2024-09-15T21:23:58.748489+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:23:58.748489+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23875584, 'num_objects': 5829, 'num_object_clones': 0, 'num_object_copies': 11658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9731, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 312, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2800}, {'shards': '0,2', 'objects': 157}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12293", 'reported_seq': 12714, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:59.580288+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:23:59.580288+0000', 'last_peered': '2024-09-15T21:23:59.580288+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:23:59.580288+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'2200", 'ondisk_log_start': "37'2200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24543232, 'num_objects': 5992, 'num_object_clones': 0, 'num_object_copies': 11984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10844, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 151, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 118}, {'shards': '2', 'objects': 2877}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:04.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'12495", 'reported_seq': 12891, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:59.564439+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:23:59.564439+0000', 'last_peered': '2024-09-15T21:23:59.564439+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:23:59.564439+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24793088, 'num_objects': 6053, 'num_object_clones': 0, 'num_object_copies': 12106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3216, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10894, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 136, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 132}, {'shards': '2', 'objects': 2839}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'12617", 'reported_seq': 13332, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.751500+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:23:58.751500+0000', 'last_peered': '2024-09-15T21:23:58.751500+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:23:58.751500+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24956928, 'num_objects': 6093, 'num_object_clones': 0, 'num_object_copies': 12186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3145, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9924, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 289, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2949}, {'shards': '0,2', 'objects': 172}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12418", 'reported_seq': 14596, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.753358+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:23:58.753358+0000', 'last_peered': '2024-09-15T21:23:58.753358+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:23:58.753358+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24551424, 'num_objects': 5994, 'num_object_clones': 0, 'num_object_copies': 11988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2362, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10003, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1020, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3632}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:04.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12401", 'reported_seq': 13040, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.694802+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:23:58.694802+0000', 'last_peered': '2024-09-15T21:23:58.694802+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:23:58.694802+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24481792, 'num_objects': 5977, 'num_object_clones': 0, 'num_object_copies': 11954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10048, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 253, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2865}, {'shards': '0,2', 'objects': 155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:04.006 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:04.717 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27038/189194 objects degraded (14.291%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94597,"data_bytes":387453588,"bytes_used":1899118592,"bytes_avail":288011173888,"bytes_total":289910292480,"degraded_objects":27038,"degraded_total":189194,"degraded_ratio":0.14291150882163281,"recovering_objects_per_sec":61,"recovering_bytes_per_sec":106813,"recovering_keys_per_sec":0,"num_objects_recovered":740,"num_bytes_recovered":1282048,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:04.753 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:04.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:05.555 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:05.555 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:05.616 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-15T21:24:04.451704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":387453588,"num_objects":94597,"num_object_clones":0,"num_object_copies":189194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94597,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173186,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13528,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1854608,"kb_used_data":921044,"kb_used_omap":12,"kb_used_meta":933491,"kb_avail":281260912,"statfs":{"total":289910292480,"available":288011173888,"internally_reserved":0,"allocated":943149056,"data_stored":942235330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":955894794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,10,3,10,4,5,1,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24346624,"num_objects":-5944,"num_object_clones":0,"num_object_copies":-11888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-427,"num_objects_misplaced":-313,"num_objects_unfound":0,"num_objects_dirty":-5944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":740,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002717"},"pg_stats":[{"pgid":"2.f","version":"50'12606","reported_seq":13359,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.690317+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:03.690317+0000","last_peered":"2024-09-15T21:24:03.690317+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:03.690317+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2873},{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12597","reported_seq":12702,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.596445+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:23:59.596445+0000","last_peered":"2024-09-15T21:23:59.596445+0000","last_clean":"2024-09-15T21:23:59.596445+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:23:59.596445+0000","last_undegraded":"2024-09-15T21:23:59.596445+0000","last_fullsized":"2024-09-15T21:23:59.596445+0000","mapping_epoch":36,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24752128,"num_objects":6043,"num_object_clones":0,"num_object_copies":12086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11323,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12812","reported_seq":19737,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.893615+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:03.893615+0000","last_peered":"2024-09-15T21:24:03.893615+0000","last_clean":"2024-09-15T21:24:03.893615+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:03.893615+0000","last_undegraded":"2024-09-15T21:24:03.893615+0000","last_fullsized":"2024-09-15T21:24:03.893615+0000","mapping_epoch":45,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809496,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11535,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12607","reported_seq":12685,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:59.490209+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:23:59.490209+0000","last_peered":"2024-09-15T21:23:59.490209+0000","last_clean":"2024-09-15T21:23:59.490209+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:23:59.490209+0000","last_undegraded":"2024-09-15T21:23:59.490209+0000","last_fullsized":"2024-09-15T21:23:59.490209+0000","mapping_epoch":36,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12607,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12760","reported_seq":12888,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.707132+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:03.707132+0000","last_peered":"2024-09-15T21:24:03.707132+0000","last_clean":"2024-09-15T21:24:03.707132+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:03.707132+0000","last_undegraded":"2024-09-15T21:24:03.707132+0000","last_fullsized":"2024-09-15T21:24:03.707132+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12558","reported_seq":13379,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.775250+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:03.775250+0000","last_peered":"2024-09-15T21:24:03.775250+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:03.775250+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23851008,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9983,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12905","reported_seq":13639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.793141+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:03.793141+0000","last_peered":"2024-09-15T21:24:03.793141+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:03.793141+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24571904,"num_objects":5999,"num_object_clones":0,"num_object_copies":11998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10541,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2829},{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12630","reported_seq":12759,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.762907+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:03.762907+0000","last_peered":"2024-09-15T21:24:03.762907+0000","last_clean":"2024-09-15T21:24:03.762907+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:03.762907+0000","last_undegraded":"2024-09-15T21:24:03.762907+0000","last_fullsized":"2024-09-15T21:24:03.762907+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10415,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12518","reported_seq":19337,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.961575+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:03.961575+0000","last_peered":"2024-09-15T21:24:03.961575+0000","last_clean":"2024-09-15T21:24:03.961575+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:03.961575+0000","last_undegraded":"2024-09-15T21:24:03.961575+0000","last_fullsized":"2024-09-15T21:24:03.961575+0000","mapping_epoch":49,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12342","reported_seq":13126,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.785271+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:03.785271+0000","last_peered":"2024-09-15T21:24:03.785271+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:03.785271+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9868,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2677},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12293","reported_seq":12714,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.580288+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:23:59.580288+0000","last_peered":"2024-09-15T21:23:59.580288+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:23:59.580288+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2200","ondisk_log_start":"37'2200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24543232,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10844,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":118},{"shards":"2","objects":2877}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12495","reported_seq":12891,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:59.564439+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:23:59.564439+0000","last_peered":"2024-09-15T21:23:59.564439+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:23:59.564439+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":132},{"shards":"2","objects":2839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12769","reported_seq":13512,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.583942+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:03.583942+0000","last_peered":"2024-09-15T21:24:03.583942+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:03.583942+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10076,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2811},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12570","reported_seq":15045,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.810946+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:03.810946+0000","last_peered":"2024-09-15T21:24:03.810946+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:03.810946+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10155,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1168,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12535","reported_seq":13190,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.652243+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:03.652243+0000","last_peered":"2024-09-15T21:24:03.652243+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:03.652243+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23932928,"num_objects":5843,"num_object_clones":0,"num_object_copies":11686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10182,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12773","reported_seq":19866,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.808067+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:03.808067+0000","last_peered":"2024-09-15T21:24:03.808067+0000","last_clean":"2024-09-15T21:24:03.808067+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:03.808067+0000","last_undegraded":"2024-09-15T21:24:03.808067+0000","last_fullsized":"2024-09-15T21:24:03.808067+0000","mapping_epoch":47,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941737,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":387453569,"num_objects":94595,"num_object_clones":0,"num_object_copies":189190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27038,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173184,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13527,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940580864,"data_stored":940517056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953553,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572784,"kb_used_data":302304,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93799056,"statfs":{"total":96636764160,"available":96050233344,"internally_reserved":0,"allocated":309559296,"data_stored":309268439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692752,"kb_used_data":336248,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93679088,"statfs":{"total":96636764160,"available":95927386112,"internally_reserved":0,"allocated":344317952,"data_stored":344031167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,9,1,7,2,5,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589072,"kb_used_data":282492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93782768,"statfs":{"total":96636764160,"available":96033554432,"internally_reserved":0,"allocated":289271808,"data_stored":288935724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342974464,"data_stored":342970985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288792576,"data_stored":288739798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308813824,"data_stored":308806273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:08.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:09.482 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:09.527 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-15T21:24:08.452706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385258132,"num_objects":94061,"num_object_clones":0,"num_object_copies":188122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173722,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13536,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160609,"ondisk_log_size":160609,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1854228,"kb_used_data":916500,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281261292,"statfs":{"total":289910292480,"available":288011563008,"internally_reserved":0,"allocated":938496000,"data_stored":937582274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":960154634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,12,5,10,6,5,0,0,5],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17801216,"num_objects":-4346,"num_object_clones":0,"num_object_copies":-8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-357,"num_objects_misplaced":-146,"num_objects_unfound":0,"num_objects_dirty":-4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4346,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":598016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"2.f","version":"50'12606","reported_seq":13359,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.690317+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:03.690317+0000","last_peered":"2024-09-15T21:24:03.690317+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:03.690317+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2873},{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12739","reported_seq":12844,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:04.509168+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:04.509168+0000","last_peered":"2024-09-15T21:24:04.509168+0000","last_clean":"2024-09-15T21:24:04.509168+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:04.509168+0000","last_undegraded":"2024-09-15T21:24:04.509168+0000","last_fullsized":"2024-09-15T21:24:04.509168+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24170496,"num_objects":5901,"num_object_clones":0,"num_object_copies":11802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12812","reported_seq":19737,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.893615+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:03.893615+0000","last_peered":"2024-09-15T21:24:03.893615+0000","last_clean":"2024-09-15T21:24:03.893615+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:03.893615+0000","last_undegraded":"2024-09-15T21:24:03.893615+0000","last_fullsized":"2024-09-15T21:24:03.893615+0000","mapping_epoch":45,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809496,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11535,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12749","reported_seq":12827,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:04.500025+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:04.500025+0000","last_peered":"2024-09-15T21:24:04.500025+0000","last_clean":"2024-09-15T21:24:04.500025+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:04.500025+0000","last_undegraded":"2024-09-15T21:24:04.500025+0000","last_fullsized":"2024-09-15T21:24:04.500025+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12749,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12760","reported_seq":12888,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.707132+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:03.707132+0000","last_peered":"2024-09-15T21:24:03.707132+0000","last_clean":"2024-09-15T21:24:03.707132+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:03.707132+0000","last_undegraded":"2024-09-15T21:24:03.707132+0000","last_fullsized":"2024-09-15T21:24:03.707132+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12558","reported_seq":13379,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.775250+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:03.775250+0000","last_peered":"2024-09-15T21:24:03.775250+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:03.775250+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23851008,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9983,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12905","reported_seq":13639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.793141+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:03.793141+0000","last_peered":"2024-09-15T21:24:03.793141+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:03.793141+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24571904,"num_objects":5999,"num_object_clones":0,"num_object_copies":11998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10541,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2829},{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12630","reported_seq":12759,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.762907+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:03.762907+0000","last_peered":"2024-09-15T21:24:03.762907+0000","last_clean":"2024-09-15T21:24:03.762907+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:03.762907+0000","last_undegraded":"2024-09-15T21:24:03.762907+0000","last_fullsized":"2024-09-15T21:24:03.762907+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10415,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12518","reported_seq":19337,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.961575+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:03.961575+0000","last_peered":"2024-09-15T21:24:03.961575+0000","last_clean":"2024-09-15T21:24:03.961575+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:03.961575+0000","last_undegraded":"2024-09-15T21:24:03.961575+0000","last_fullsized":"2024-09-15T21:24:03.961575+0000","mapping_epoch":49,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12342","reported_seq":13126,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.785271+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:03.785271+0000","last_peered":"2024-09-15T21:24:03.785271+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:03.785271+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9868,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2677},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12412","reported_seq":12839,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:04.483061+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:04.483061+0000","last_peered":"2024-09-15T21:24:04.483061+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:04.483061+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12628","reported_seq":13034,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:04.390806+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:04.390806+0000","last_peered":"2024-09-15T21:24:04.390806+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:04.390806+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11027,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2711}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12769","reported_seq":13512,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.583942+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:03.583942+0000","last_peered":"2024-09-15T21:24:03.583942+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:03.583942+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10076,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2811},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12570","reported_seq":15045,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.810946+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:03.810946+0000","last_peered":"2024-09-15T21:24:03.810946+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:03.810946+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10155,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1168,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12535","reported_seq":13190,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.652243+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:03.652243+0000","last_peered":"2024-09-15T21:24:03.652243+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:03.652243+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23932928,"num_objects":5843,"num_object_clones":0,"num_object_copies":11686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10182,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12773","reported_seq":19866,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.808067+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:03.808067+0000","last_peered":"2024-09-15T21:24:03.808067+0000","last_clean":"2024-09-15T21:24:03.808067+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:03.808067+0000","last_undegraded":"2024-09-15T21:24:03.808067+0000","last_fullsized":"2024-09-15T21:24:03.808067+0000","mapping_epoch":47,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941737,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385258113,"num_objects":94059,"num_object_clones":0,"num_object_copies":188118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173720,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13535,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936079360,"data_stored":936015552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160606,"ondisk_log_size":160606,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953554,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572404,"kb_used_data":297760,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93799436,"statfs":{"total":96636764160,"available":96050622464,"internally_reserved":0,"allocated":304906240,"data_stored":304615383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692752,"kb_used_data":336248,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93679088,"statfs":{"total":96636764160,"available":95927386112,"internally_reserved":0,"allocated":344317952,"data_stored":344031167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,9,1,7,2,5,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589072,"kb_used_data":282492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93782768,"statfs":{"total":96636764160,"available":96033554432,"internally_reserved":0,"allocated":289271808,"data_stored":288935724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342974464,"data_stored":342970985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288792576,"data_stored":288739798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304312320,"data_stored":304304769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:09.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:10.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:10.082 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:10.150 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-15T21:24:08.452706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385258132,"num_objects":94061,"num_object_clones":0,"num_object_copies":188122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173722,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13536,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160609,"ondisk_log_size":160609,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1854228,"kb_used_data":916500,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":281261292,"statfs":{"total":289910292480,"available":288011563008,"internally_reserved":0,"allocated":938496000,"data_stored":937582274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":960154634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,12,5,10,6,5,0,0,5],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17801216,"num_objects":-4346,"num_object_clones":0,"num_object_copies":-8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-357,"num_objects_misplaced":-146,"num_objects_unfound":0,"num_objects_dirty":-4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4346,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":598016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002802"},"pg_stats":[{"pgid":"2.f","version":"50'12606","reported_seq":13359,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.690317+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:03.690317+0000","last_peered":"2024-09-15T21:24:03.690317+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:03.690317+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12606,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2873},{"shards":"0,2","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12739","reported_seq":12844,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:04.509168+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:04.509168+0000","last_peered":"2024-09-15T21:24:04.509168+0000","last_clean":"2024-09-15T21:24:04.509168+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:04.509168+0000","last_undegraded":"2024-09-15T21:24:04.509168+0000","last_fullsized":"2024-09-15T21:24:04.509168+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24170496,"num_objects":5901,"num_object_clones":0,"num_object_copies":11802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12812","reported_seq":19737,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.893615+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:03.893615+0000","last_peered":"2024-09-15T21:24:03.893615+0000","last_clean":"2024-09-15T21:24:03.893615+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:03.893615+0000","last_undegraded":"2024-09-15T21:24:03.893615+0000","last_fullsized":"2024-09-15T21:24:03.893615+0000","mapping_epoch":45,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24809496,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11535,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12749","reported_seq":12827,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:04.500025+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:04.500025+0000","last_peered":"2024-09-15T21:24:04.500025+0000","last_clean":"2024-09-15T21:24:04.500025+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:04.500025+0000","last_undegraded":"2024-09-15T21:24:04.500025+0000","last_fullsized":"2024-09-15T21:24:04.500025+0000","mapping_epoch":36,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12749,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12760","reported_seq":12888,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.707132+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:03.707132+0000","last_peered":"2024-09-15T21:24:03.707132+0000","last_clean":"2024-09-15T21:24:03.707132+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:03.707132+0000","last_undegraded":"2024-09-15T21:24:03.707132+0000","last_fullsized":"2024-09-15T21:24:03.707132+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24424448,"num_objects":5963,"num_object_clones":0,"num_object_copies":11926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12558","reported_seq":13379,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.775250+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:03.775250+0000","last_peered":"2024-09-15T21:24:03.775250+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:03.775250+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23851008,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9983,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'12905","reported_seq":13639,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.793141+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:03.793141+0000","last_peered":"2024-09-15T21:24:03.793141+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:03.793141+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24571904,"num_objects":5999,"num_object_clones":0,"num_object_copies":11998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10541,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2829},{"shards":"0,2","objects":19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12630","reported_seq":12759,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.762907+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:03.762907+0000","last_peered":"2024-09-15T21:24:03.762907+0000","last_clean":"2024-09-15T21:24:03.762907+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:03.762907+0000","last_undegraded":"2024-09-15T21:24:03.762907+0000","last_fullsized":"2024-09-15T21:24:03.762907+0000","mapping_epoch":35,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10415,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12518","reported_seq":19337,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.961575+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:03.961575+0000","last_peered":"2024-09-15T21:24:03.961575+0000","last_clean":"2024-09-15T21:24:03.961575+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:03.961575+0000","last_undegraded":"2024-09-15T21:24:03.961575+0000","last_fullsized":"2024-09-15T21:24:03.961575+0000","mapping_epoch":49,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12342","reported_seq":13126,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.785271+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:03.785271+0000","last_peered":"2024-09-15T21:24:03.785271+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:03.785271+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2300","ondisk_log_start":"37'2300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9868,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2677},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12412","reported_seq":12839,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:04.483061+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:04.483061+0000","last_peered":"2024-09-15T21:24:04.483061+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:04.483061+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2400","ondisk_log_start":"37'2400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12628","reported_seq":13034,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:04.390806+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:04.390806+0000","last_peered":"2024-09-15T21:24:04.390806+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:04.390806+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11027,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2711}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12769","reported_seq":13512,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.583942+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:03.583942+0000","last_peered":"2024-09-15T21:24:03.583942+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:03.583942+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10076,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2811},{"shards":"0,2","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12570","reported_seq":15045,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.810946+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:03.810946+0000","last_peered":"2024-09-15T21:24:03.810946+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:03.810946+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10155,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1168,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12535","reported_seq":13190,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.652243+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:03.652243+0000","last_peered":"2024-09-15T21:24:03.652243+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:03.652243+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23932928,"num_objects":5843,"num_object_clones":0,"num_object_copies":11686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10182,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2739},{"shards":"0,2","objects":21}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12773","reported_seq":19866,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.808067+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:03.808067+0000","last_peered":"2024-09-15T21:24:03.808067+0000","last_clean":"2024-09-15T21:24:03.808067+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:03.808067+0000","last_undegraded":"2024-09-15T21:24:03.808067+0000","last_fullsized":"2024-09-15T21:24:03.808067+0000","mapping_epoch":47,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23941737,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10433,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385258113,"num_objects":94059,"num_object_clones":0,"num_object_copies":188118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27030,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173720,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13535,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936079360,"data_stored":936015552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160606,"ondisk_log_size":160606,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953554,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572404,"kb_used_data":297760,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93799436,"statfs":{"total":96636764160,"available":96050622464,"internally_reserved":0,"allocated":304906240,"data_stored":304615383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149782,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692752,"kb_used_data":336248,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93679088,"statfs":{"total":96636764160,"available":95927386112,"internally_reserved":0,"allocated":344317952,"data_stored":344031167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,9,1,7,2,5,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589072,"kb_used_data":282492,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93782768,"statfs":{"total":96636764160,"available":96033554432,"internally_reserved":0,"allocated":289271808,"data_stored":288935724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":313913004},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342974464,"data_stored":342970985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288792576,"data_stored":288739798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304312320,"data_stored":304304769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:10.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:10.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12606", 'reported_seq': 13359, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.690317+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:03.690317+0000', 'last_peered': '2024-09-15T21:24:03.690317+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:03.690317+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24190976, 'num_objects': 5906, 'num_object_clones': 0, 'num_object_copies': 11812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3033, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12606, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 334, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2873}, {'shards': '0,2', 'objects': 17}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:10.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12558", 'reported_seq': 13379, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.775250+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:03.775250+0000', 'last_peered': '2024-09-15T21:24:03.775250+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:03.775250+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23851008, 'num_objects': 5823, 'num_object_clones': 0, 'num_object_copies': 11646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9983, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 341, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2793}, {'shards': '0,2', 'objects': 6}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'12905", 'reported_seq': 13639, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.793141+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:03.793141+0000', 'last_peered': '2024-09-15T21:24:03.793141+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:03.793141+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "37'2900", 'ondisk_log_start': "37'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24571904, 'num_objects': 5999, 'num_object_clones': 0, 'num_object_copies': 11998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3172, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10541, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 301, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2829}, {'shards': '0,2', 'objects': 19}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12342", 'reported_seq': 13126, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.785271+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:03.785271+0000', 'last_peered': '2024-09-15T21:24:03.785271+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:03.785271+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'2300", 'ondisk_log_start': "37'2300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23314432, 'num_objects': 5692, 'num_object_clones': 0, 'num_object_copies': 11384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9868, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2677}, {'shards': '0,2', 'objects': 20}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12412", 'reported_seq': 12839, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:04.483061+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:04.483061+0000', 'last_peered': '2024-09-15T21:24:04.483061+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:04.483061+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'2400", 'ondisk_log_start': "37'2400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24055808, 'num_objects': 5873, 'num_object_clones': 0, 'num_object_copies': 11746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10963, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2761}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'12628", 'reported_seq': 13034, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:04.390806+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:04.390806+0000', 'last_peered': '2024-09-15T21:24:04.390806+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:04.390806+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24248320, 'num_objects': 5920, 'num_object_clones': 0, 'num_object_copies': 11840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3211, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11027, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 141, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2711}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:10.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:10.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'12769", 'reported_seq': 13512, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.583942+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:03.583942+0000', 'last_peered': '2024-09-15T21:24:03.583942+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:03.583942+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24334336, 'num_objects': 5941, 'num_object_clones': 0, 'num_object_copies': 11882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10076, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2811}, {'shards': '0,2', 'objects': 20}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:10.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12570", 'reported_seq': 15045, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.810946+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:03.810946+0000', 'last_peered': '2024-09-15T21:24:03.810946+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:03.810946+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23928832, 'num_objects': 5842, 'num_object_clones': 0, 'num_object_copies': 11684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2214, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10155, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1168, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:10.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12535", 'reported_seq': 13190, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.652243+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:03.652243+0000', 'last_peered': '2024-09-15T21:24:03.652243+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:03.652243+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2500", 'ondisk_log_start': "37'2500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23932928, 'num_objects': 5843, 'num_object_clones': 0, 'num_object_copies': 11686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3106, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10182, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 261, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2739}, {'shards': '0,2', 'objects': 21}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:10.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:10.825 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:10.825 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26865/183332 objects degraded (14.654%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91666,"data_bytes":375448212,"bytes_used":1867653120,"bytes_avail":288042639360,"bytes_total":289910292480,"degraded_objects":26865,"degraded_total":183332,"degraded_ratio":0.14653742936312264,"recovering_objects_per_sec":55,"recovering_bytes_per_sec":49823,"recovering_keys_per_sec":0,"num_objects_recovered":668,"num_bytes_recovered":598016,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":561},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:10.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:10.874 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:11.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:11.671 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:11.720 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-15T21:24:10.453130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375448212,"num_objects":91666,"num_object_clones":0,"num_object_copies":183332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176117,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13701,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160704,"ondisk_log_size":160704,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1823880,"kb_used_data":898624,"kb_used_omap":12,"kb_used_meta":925171,"kb_avail":281291640,"statfs":{"total":289910292480,"available":288042639360,"internally_reserved":0,"allocated":920190976,"data_stored":919277250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":947375114},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,8,4,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27611136,"num_objects":-6741,"num_object_clones":0,"num_object_copies":-13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-522,"num_objects_misplaced":-146,"num_objects_unfound":0,"num_objects_dirty":-6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6741,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":668,"num_bytes_recovered":598016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002784"},"pg_stats":[{"pgid":"2.f","version":"50'12732","reported_seq":13489,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.789363+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:08.789363+0000","last_peered":"2024-09-15T21:24:08.789363+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:08.789363+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23674880,"num_objects":5780,"num_object_clones":0,"num_object_copies":11560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12732,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2749}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12876","reported_seq":12981,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.550564+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:09.550564+0000","last_peered":"2024-09-15T21:24:09.550564+0000","last_clean":"2024-09-15T21:24:09.550564+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:09.550564+0000","last_undegraded":"2024-09-15T21:24:09.550564+0000","last_fullsized":"2024-09-15T21:24:09.550564+0000","mapping_epoch":36,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23609344,"num_objects":5764,"num_object_clones":0,"num_object_copies":11528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'12959","reported_seq":19884,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.037446+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:09.037446+0000","last_peered":"2024-09-15T21:24:09.037446+0000","last_clean":"2024-09-15T21:24:09.037446+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:09.037446+0000","last_undegraded":"2024-09-15T21:24:09.037446+0000","last_fullsized":"2024-09-15T21:24:09.037446+0000","mapping_epoch":45,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24207384,"num_objects":5911,"num_object_clones":0,"num_object_copies":11822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11682,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12906","reported_seq":12984,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.622198+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:09.622198+0000","last_peered":"2024-09-15T21:24:09.622198+0000","last_clean":"2024-09-15T21:24:09.622198+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:09.622198+0000","last_undegraded":"2024-09-15T21:24:09.622198+0000","last_fullsized":"2024-09-15T21:24:09.622198+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12906,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'12920","reported_seq":13048,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.753384+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:08.753384+0000","last_peered":"2024-09-15T21:24:08.753384+0000","last_clean":"2024-09-15T21:24:08.753384+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:08.753384+0000","last_undegraded":"2024-09-15T21:24:08.753384+0000","last_fullsized":"2024-09-15T21:24:08.753384+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23769088,"num_objects":5803,"num_object_clones":0,"num_object_copies":11606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10778,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12704","reported_seq":13527,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.765525+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:08.765525+0000","last_peered":"2024-09-15T21:24:08.765525+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:08.765525+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23252992,"num_objects":5677,"num_object_clones":0,"num_object_copies":11354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10129,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13076","reported_seq":13822,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.830819+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:08.830819+0000","last_peered":"2024-09-15T21:24:08.830819+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:08.830819+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2664}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12773","reported_seq":12902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.827210+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:08.827210+0000","last_peered":"2024-09-15T21:24:08.827210+0000","last_clean":"2024-09-15T21:24:08.827210+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:08.827210+0000","last_undegraded":"2024-09-15T21:24:08.827210+0000","last_fullsized":"2024-09-15T21:24:08.827210+0000","mapping_epoch":35,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23064576,"num_objects":5631,"num_object_clones":0,"num_object_copies":11262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12661","reported_seq":19484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.090029+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:09.090029+0000","last_peered":"2024-09-15T21:24:09.090029+0000","last_clean":"2024-09-15T21:24:09.090029+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:09.090029+0000","last_undegraded":"2024-09-15T21:24:09.090029+0000","last_fullsized":"2024-09-15T21:24:09.090029+0000","mapping_epoch":49,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23404544,"num_objects":5714,"num_object_clones":0,"num_object_copies":11428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11247,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12504","reported_seq":13285,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.873696+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:08.873696+0000","last_peered":"2024-09-15T21:24:08.873696+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:08.873696+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10030,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12587","reported_seq":13010,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:09.570598+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:09.570598+0000","last_peered":"2024-09-15T21:24:09.570598+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:09.570598+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339008,"num_objects":5698,"num_object_clones":0,"num_object_copies":11396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11138,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12762","reported_seq":13174,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:09.567945+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:09.567945+0000","last_peered":"2024-09-15T21:24:09.567945+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:09.567945+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11161,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'12911","reported_seq":13653,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.762072+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:08.762072+0000","last_peered":"2024-09-15T21:24:08.762072+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:08.762072+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10218,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12717","reported_seq":15495,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.827676+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:08.827676+0000","last_peered":"2024-09-15T21:24:08.827676+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:08.827676+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12677","reported_seq":13336,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.856877+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:08.856877+0000","last_peered":"2024-09-15T21:24:08.856877+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:08.856877+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23351296,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10324,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'12936","reported_seq":20028,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.090566+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:09.090566+0000","last_peered":"2024-09-15T21:24:09.090566+0000","last_clean":"2024-09-15T21:24:09.090566+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:09.090566+0000","last_undegraded":"2024-09-15T21:24:09.090566+0000","last_fullsized":"2024-09-15T21:24:09.090566+0000","mapping_epoch":47,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23274089,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10596,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375448193,"num_objects":91664,"num_object_clones":0,"num_object_copies":183328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26865,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176115,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13700,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916787200,"data_stored":916723392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953555,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555368,"kb_used_data":293208,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93816472,"statfs":{"total":96636764160,"available":96068067328,"internally_reserved":0,"allocated":300244992,"data_stored":299954135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694308,"kb_used_data":329472,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93677532,"statfs":{"total":96636764160,"available":95925792768,"internally_reserved":0,"allocated":337379328,"data_stored":337092543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":373550769},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574204,"kb_used_data":275944,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93797636,"statfs":{"total":96636764160,"available":96048779264,"internally_reserved":0,"allocated":282566656,"data_stored":282230572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":305393324},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335630336,"data_stored":335626857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281812992,"data_stored":281760214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299343872,"data_stored":299336321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:14.722 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:15.548 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:15.549 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:15.635 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-15T21:24:14.457675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368718484,"num_objects":90023,"num_object_clones":0,"num_object_copies":180046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177760,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13855,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824536,"kb_used_data":885400,"kb_used_omap":12,"kb_used_meta":939059,"kb_avail":281290984,"statfs":{"total":289910292480,"available":288041967616,"internally_reserved":0,"allocated":906649600,"data_stored":905735874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":961596426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,0,1,2,1,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25841664,"num_objects":-6309,"num_object_clones":0,"num_object_copies":-12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6309,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006385"},"pg_stats":[{"pgid":"2.f","version":"50'12876","reported_seq":13633,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.859753+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:13.859753+0000","last_peered":"2024-09-15T21:24:13.859753+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:13.859753+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12876,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'12876","reported_seq":12981,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.550564+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:09.550564+0000","last_peered":"2024-09-15T21:24:09.550564+0000","last_clean":"2024-09-15T21:24:09.550564+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:09.550564+0000","last_undegraded":"2024-09-15T21:24:09.550564+0000","last_fullsized":"2024-09-15T21:24:09.550564+0000","mapping_epoch":36,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23609344,"num_objects":5764,"num_object_clones":0,"num_object_copies":11528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13076","reported_seq":20001,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.166493+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:14.166493+0000","last_peered":"2024-09-15T21:24:14.166493+0000","last_clean":"2024-09-15T21:24:14.166493+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:14.166493+0000","last_undegraded":"2024-09-15T21:24:14.166493+0000","last_fullsized":"2024-09-15T21:24:14.166493+0000","mapping_epoch":45,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728152,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11799,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'12906","reported_seq":12984,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:09.622198+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:09.622198+0000","last_peered":"2024-09-15T21:24:09.622198+0000","last_clean":"2024-09-15T21:24:09.622198+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:09.622198+0000","last_undegraded":"2024-09-15T21:24:09.622198+0000","last_fullsized":"2024-09-15T21:24:09.622198+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12906,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13060","reported_seq":13188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.873515+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:13.873515+0000","last_peered":"2024-09-15T21:24:13.873515+0000","last_clean":"2024-09-15T21:24:13.873515+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:13.873515+0000","last_undegraded":"2024-09-15T21:24:13.873515+0000","last_fullsized":"2024-09-15T21:24:13.873515+0000","mapping_epoch":35,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23195648,"num_objects":5663,"num_object_clones":0,"num_object_copies":11326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12836","reported_seq":13659,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.885164+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:13.885164+0000","last_peered":"2024-09-15T21:24:13.885164+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:13.885164+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13214","reported_seq":13960,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.826406+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:13.826406+0000","last_peered":"2024-09-15T21:24:13.826406+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:13.826406+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12904","reported_seq":13033,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.798204+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:13.798204+0000","last_peered":"2024-09-15T21:24:13.798204+0000","last_clean":"2024-09-15T21:24:13.798204+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:13.798204+0000","last_undegraded":"2024-09-15T21:24:13.798204+0000","last_fullsized":"2024-09-15T21:24:13.798204+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12812","reported_seq":19635,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.906610+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:13.906610+0000","last_peered":"2024-09-15T21:24:13.906610+0000","last_clean":"2024-09-15T21:24:13.906610+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:13.906610+0000","last_undegraded":"2024-09-15T21:24:13.906610+0000","last_fullsized":"2024-09-15T21:24:13.906610+0000","mapping_epoch":49,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12652","reported_seq":13436,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.884966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:13.884966+0000","last_peered":"2024-09-15T21:24:13.884966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:13.884966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12587","reported_seq":13010,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:09.570598+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:09.570598+0000","last_peered":"2024-09-15T21:24:09.570598+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:09.570598+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2500","ondisk_log_start":"37'2500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339008,"num_objects":5698,"num_object_clones":0,"num_object_copies":11396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11138,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2586}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12762","reported_seq":13174,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:09.567945+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:09.567945+0000","last_peered":"2024-09-15T21:24:09.567945+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:09.567945+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23699456,"num_objects":5786,"num_object_clones":0,"num_object_copies":11572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11161,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13059","reported_seq":13801,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.860822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:13.860822+0000","last_peered":"2024-09-15T21:24:13.860822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:13.860822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23146496,"num_objects":5651,"num_object_clones":0,"num_object_copies":11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10366,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12869","reported_seq":15955,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.891830+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:13.891830+0000","last_peered":"2024-09-15T21:24:13.891830+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:13.891830+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12802","reported_seq":13461,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.763793+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:13.763793+0000","last_peered":"2024-09-15T21:24:13.763793+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:13.763793+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22839296,"num_objects":5576,"num_object_clones":0,"num_object_copies":11152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13053","reported_seq":20147,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.870672+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:13.870672+0000","last_peered":"2024-09-15T21:24:13.870672+0000","last_clean":"2024-09-15T21:24:13.870672+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:13.870672+0000","last_undegraded":"2024-09-15T21:24:13.870672+0000","last_fullsized":"2024-09-15T21:24:13.870672+0000","mapping_epoch":47,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22794857,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":368718465,"num_objects":90021,"num_object_clones":0,"num_object_copies":180042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90021,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177758,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13854,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":903311360,"data_stored":903247552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160744,"ondisk_log_size":160744,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953555,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555368,"kb_used_data":293208,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93816472,"statfs":{"total":96636764160,"available":96068067328,"internally_reserved":0,"allocated":300244992,"data_stored":299954135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675148,"kb_used_data":322804,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93696692,"statfs":{"total":96636764160,"available":95945412608,"internally_reserved":0,"allocated":330551296,"data_stored":330264511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594020,"kb_used_data":269388,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777820,"statfs":{"total":96636764160,"available":96028487680,"internally_reserved":0,"allocated":275853312,"data_stored":275517228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328900608,"data_stored":328897129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275066880,"data_stored":275014102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299343872,"data_stored":299336321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:15.636 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:16.589 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:16.646 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-15T21:24:16.458454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366445204,"num_objects":89468,"num_object_clones":0,"num_object_copies":178936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178315,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13855,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824296,"kb_used_data":881000,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":281291224,"statfs":{"total":289910292480,"available":288042213376,"internally_reserved":0,"allocated":902144000,"data_stored":901230274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":965856266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,2,2,2,6,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21008384,"num_objects":-5129,"num_object_clones":0,"num_object_copies":-10258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006749"},"pg_stats":[{"pgid":"2.f","version":"50'12876","reported_seq":13633,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.859753+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:13.859753+0000","last_peered":"2024-09-15T21:24:13.859753+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:13.859753+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12876,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13000","reported_seq":13105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.628733+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:14.628733+0000","last_peered":"2024-09-15T21:24:14.628733+0000","last_clean":"2024-09-15T21:24:14.628733+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:14.628733+0000","last_undegraded":"2024-09-15T21:24:14.628733+0000","last_fullsized":"2024-09-15T21:24:14.628733+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23101440,"num_objects":5640,"num_object_clones":0,"num_object_copies":11280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11726,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13076","reported_seq":20001,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.166493+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:14.166493+0000","last_peered":"2024-09-15T21:24:14.166493+0000","last_clean":"2024-09-15T21:24:14.166493+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:14.166493+0000","last_undegraded":"2024-09-15T21:24:14.166493+0000","last_fullsized":"2024-09-15T21:24:14.166493+0000","mapping_epoch":45,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728152,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11799,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13046","reported_seq":13124,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.565596+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:14.565596+0000","last_peered":"2024-09-15T21:24:14.565596+0000","last_clean":"2024-09-15T21:24:14.565596+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:14.565596+0000","last_undegraded":"2024-09-15T21:24:14.565596+0000","last_fullsized":"2024-09-15T21:24:14.565596+0000","mapping_epoch":36,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13046,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13060","reported_seq":13188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.873515+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:13.873515+0000","last_peered":"2024-09-15T21:24:13.873515+0000","last_clean":"2024-09-15T21:24:13.873515+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:13.873515+0000","last_undegraded":"2024-09-15T21:24:13.873515+0000","last_fullsized":"2024-09-15T21:24:13.873515+0000","mapping_epoch":35,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23195648,"num_objects":5663,"num_object_clones":0,"num_object_copies":11326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12836","reported_seq":13659,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.885164+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:13.885164+0000","last_peered":"2024-09-15T21:24:13.885164+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:13.885164+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13214","reported_seq":13960,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.826406+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:13.826406+0000","last_peered":"2024-09-15T21:24:13.826406+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:13.826406+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12904","reported_seq":13033,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.798204+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:13.798204+0000","last_peered":"2024-09-15T21:24:13.798204+0000","last_clean":"2024-09-15T21:24:13.798204+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:13.798204+0000","last_undegraded":"2024-09-15T21:24:13.798204+0000","last_fullsized":"2024-09-15T21:24:13.798204+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12812","reported_seq":19635,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.906610+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:13.906610+0000","last_peered":"2024-09-15T21:24:13.906610+0000","last_clean":"2024-09-15T21:24:13.906610+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:13.906610+0000","last_undegraded":"2024-09-15T21:24:13.906610+0000","last_fullsized":"2024-09-15T21:24:13.906610+0000","mapping_epoch":49,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12652","reported_seq":13436,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.884966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:13.884966+0000","last_peered":"2024-09-15T21:24:13.884966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:13.884966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12737","reported_seq":13164,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:14.560720+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:14.560720+0000","last_peered":"2024-09-15T21:24:14.560720+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:14.560720+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22724608,"num_objects":5548,"num_object_clones":0,"num_object_copies":11096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11288,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12903","reported_seq":13315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:14.531389+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:14.531389+0000","last_peered":"2024-09-15T21:24:14.531389+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:14.531389+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23121920,"num_objects":5645,"num_object_clones":0,"num_object_copies":11290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11302,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13059","reported_seq":13801,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.860822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:13.860822+0000","last_peered":"2024-09-15T21:24:13.860822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:13.860822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23146496,"num_objects":5651,"num_object_clones":0,"num_object_copies":11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10366,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12869","reported_seq":15955,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.891830+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:13.891830+0000","last_peered":"2024-09-15T21:24:13.891830+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:13.891830+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12802","reported_seq":13461,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.763793+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:13.763793+0000","last_peered":"2024-09-15T21:24:13.763793+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:13.763793+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22839296,"num_objects":5576,"num_object_clones":0,"num_object_copies":11152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13053","reported_seq":20147,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.870672+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:13.870672+0000","last_peered":"2024-09-15T21:24:13.870672+0000","last_clean":"2024-09-15T21:24:13.870672+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:13.870672+0000","last_undegraded":"2024-09-15T21:24:13.870672+0000","last_fullsized":"2024-09-15T21:24:13.870672+0000","mapping_epoch":47,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22794857,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366445185,"num_objects":89466,"num_object_clones":0,"num_object_copies":178932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178313,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13854,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":898953216,"data_stored":898889408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953556,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555128,"kb_used_data":288808,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93816712,"statfs":{"total":96636764160,"available":96068313088,"internally_reserved":0,"allocated":295739392,"data_stored":295448535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675148,"kb_used_data":322804,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93696692,"statfs":{"total":96636764160,"available":95945412608,"internally_reserved":0,"allocated":330551296,"data_stored":330264511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594020,"kb_used_data":269388,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777820,"statfs":{"total":96636764160,"available":96028487680,"internally_reserved":0,"allocated":275853312,"data_stored":275517228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328900608,"data_stored":328897129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275066880,"data_stored":275014102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294985728,"data_stored":294978177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'12876", 'reported_seq': 13633, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.859753+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:13.859753+0000', 'last_peered': '2024-09-15T21:24:13.859753+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:13.859753+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23085056, 'num_objects': 5636, 'num_object_clones': 0, 'num_object_copies': 11272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12876, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2605}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12836", 'reported_seq': 13659, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.885164+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:13.885164+0000', 'last_peered': '2024-09-15T21:24:13.885164+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:13.885164+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22712320, 'num_objects': 5545, 'num_object_clones': 0, 'num_object_copies': 11090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10261, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2516}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'13214", 'reported_seq': 13960, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.826406+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:13.826406+0000', 'last_peered': '2024-09-15T21:24:13.826406+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:13.826406+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "41'3200", 'ondisk_log_start': "41'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23306240, 'num_objects': 5690, 'num_object_clones': 0, 'num_object_copies': 11380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10850, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:16.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12652", 'reported_seq': 13436, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.884966+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:13.884966+0000', 'last_peered': '2024-09-15T21:24:13.884966+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:13.884966+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'2600", 'ondisk_log_start': "37'2600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22044672, 'num_objects': 5382, 'num_object_clones': 0, 'num_object_copies': 10764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5382, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10178, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2367}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12737", 'reported_seq': 13164, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:14.560720+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:14.560720+0000', 'last_peered': '2024-09-15T21:24:14.560720+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:14.560720+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22724608, 'num_objects': 5548, 'num_object_clones': 0, 'num_object_copies': 11096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11288, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2436}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'12903", 'reported_seq': 13315, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:14.531389+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:14.531389+0000', 'last_peered': '2024-09-15T21:24:14.531389+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:14.531389+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "40'2900", 'ondisk_log_start': "40'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23121920, 'num_objects': 5645, 'num_object_clones': 0, 'num_object_copies': 11290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11302, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2439}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13059", 'reported_seq': 13801, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.860822+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:13.860822+0000', 'last_peered': '2024-09-15T21:24:13.860822+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:13.860822+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "40'3000", 'ondisk_log_start': "40'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23146496, 'num_objects': 5651, 'num_object_clones': 0, 'num_object_copies': 11302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10366, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2521}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'12869", 'reported_seq': 15955, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.891830+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:13.891830+0000', 'last_peered': '2024-09-15T21:24:13.891830+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:13.891830+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22704128, 'num_objects': 5543, 'num_object_clones': 0, 'num_object_copies': 11086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1909, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5543, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10454, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1473, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3634}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:16.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12802", 'reported_seq': 13461, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:13.763793+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:13.763793+0000', 'last_peered': '2024-09-15T21:24:13.763793+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:13.763793+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "37'2800", 'ondisk_log_start': "37'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22839296, 'num_objects': 5576, 'num_object_clones': 0, 'num_object_copies': 11152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10449, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2474}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:16.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:17.185 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:17.185 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26711/178936 objects degraded (14.928%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89468,"data_bytes":366445204,"bytes_used":1868079104,"bytes_avail":288042213376,"bytes_total":289910292480,"degraded_objects":26711,"degraded_total":178936,"degraded_ratio":0.14927683641078374,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":427},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-15T21:20:12.350481+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:17.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:17.234 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:17.994 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:17.996 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:18.075 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-15T21:24:16.458454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366445204,"num_objects":89468,"num_object_clones":0,"num_object_copies":178936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178315,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13855,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824296,"kb_used_data":881000,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":281291224,"statfs":{"total":289910292480,"available":288042213376,"internally_reserved":0,"allocated":902144000,"data_stored":901230274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":965856266},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,2,2,2,6,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21008384,"num_objects":-5129,"num_object_clones":0,"num_object_copies":-10258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006749"},"pg_stats":[{"pgid":"2.f","version":"50'12876","reported_seq":13633,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.859753+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:13.859753+0000","last_peered":"2024-09-15T21:24:13.859753+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:13.859753+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12876,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13000","reported_seq":13105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.628733+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:14.628733+0000","last_peered":"2024-09-15T21:24:14.628733+0000","last_clean":"2024-09-15T21:24:14.628733+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:14.628733+0000","last_undegraded":"2024-09-15T21:24:14.628733+0000","last_fullsized":"2024-09-15T21:24:14.628733+0000","mapping_epoch":36,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23101440,"num_objects":5640,"num_object_clones":0,"num_object_copies":11280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11726,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13076","reported_seq":20001,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.166493+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:14.166493+0000","last_peered":"2024-09-15T21:24:14.166493+0000","last_clean":"2024-09-15T21:24:14.166493+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:14.166493+0000","last_undegraded":"2024-09-15T21:24:14.166493+0000","last_fullsized":"2024-09-15T21:24:14.166493+0000","mapping_epoch":45,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728152,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11799,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13046","reported_seq":13124,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:14.565596+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:14.565596+0000","last_peered":"2024-09-15T21:24:14.565596+0000","last_clean":"2024-09-15T21:24:14.565596+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:14.565596+0000","last_undegraded":"2024-09-15T21:24:14.565596+0000","last_fullsized":"2024-09-15T21:24:14.565596+0000","mapping_epoch":36,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13046,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13060","reported_seq":13188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.873515+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:13.873515+0000","last_peered":"2024-09-15T21:24:13.873515+0000","last_clean":"2024-09-15T21:24:13.873515+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:13.873515+0000","last_undegraded":"2024-09-15T21:24:13.873515+0000","last_fullsized":"2024-09-15T21:24:13.873515+0000","mapping_epoch":35,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23195648,"num_objects":5663,"num_object_clones":0,"num_object_copies":11326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10918,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12836","reported_seq":13659,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.885164+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:13.885164+0000","last_peered":"2024-09-15T21:24:13.885164+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:13.885164+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10261,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13214","reported_seq":13960,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.826406+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:13.826406+0000","last_peered":"2024-09-15T21:24:13.826406+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:13.826406+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10850,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'12904","reported_seq":13033,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.798204+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:13.798204+0000","last_peered":"2024-09-15T21:24:13.798204+0000","last_clean":"2024-09-15T21:24:13.798204+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:13.798204+0000","last_undegraded":"2024-09-15T21:24:13.798204+0000","last_fullsized":"2024-09-15T21:24:13.798204+0000","mapping_epoch":35,"log_start":"37'2900","ondisk_log_start":"37'2900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10689,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12812","reported_seq":19635,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.906610+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:13.906610+0000","last_peered":"2024-09-15T21:24:13.906610+0000","last_clean":"2024-09-15T21:24:13.906610+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:13.906610+0000","last_undegraded":"2024-09-15T21:24:13.906610+0000","last_fullsized":"2024-09-15T21:24:13.906610+0000","mapping_epoch":49,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12652","reported_seq":13436,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.884966+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:13.884966+0000","last_peered":"2024-09-15T21:24:13.884966+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:13.884966+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2600","ondisk_log_start":"37'2600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22044672,"num_objects":5382,"num_object_clones":0,"num_object_copies":10764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12737","reported_seq":13164,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:14.560720+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:14.560720+0000","last_peered":"2024-09-15T21:24:14.560720+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:14.560720+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22724608,"num_objects":5548,"num_object_clones":0,"num_object_copies":11096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11288,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'12903","reported_seq":13315,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:14.531389+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:14.531389+0000","last_peered":"2024-09-15T21:24:14.531389+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:14.531389+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23121920,"num_objects":5645,"num_object_clones":0,"num_object_copies":11290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11302,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13059","reported_seq":13801,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.860822+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:13.860822+0000","last_peered":"2024-09-15T21:24:13.860822+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:13.860822+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23146496,"num_objects":5651,"num_object_clones":0,"num_object_copies":11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10366,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'12869","reported_seq":15955,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.891830+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:13.891830+0000","last_peered":"2024-09-15T21:24:13.891830+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:13.891830+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12802","reported_seq":13461,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.763793+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:13.763793+0000","last_peered":"2024-09-15T21:24:13.763793+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:13.763793+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"37'2800","ondisk_log_start":"37'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22839296,"num_objects":5576,"num_object_clones":0,"num_object_copies":11152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10449,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13053","reported_seq":20147,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.870672+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:13.870672+0000","last_peered":"2024-09-15T21:24:13.870672+0000","last_clean":"2024-09-15T21:24:13.870672+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:13.870672+0000","last_undegraded":"2024-09-15T21:24:13.870672+0000","last_fullsized":"2024-09-15T21:24:13.870672+0000","mapping_epoch":47,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22794857,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366445185,"num_objects":89466,"num_object_clones":0,"num_object_copies":178932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89466,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178313,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13854,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":898953216,"data_stored":898889408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953556,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555128,"kb_used_data":288808,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93816712,"statfs":{"total":96636764160,"available":96068313088,"internally_reserved":0,"allocated":295739392,"data_stored":295448535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149784,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":675148,"kb_used_data":322804,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93696692,"statfs":{"total":96636764160,"available":95945412608,"internally_reserved":0,"allocated":330551296,"data_stored":330264511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":360771249},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594020,"kb_used_data":269388,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777820,"statfs":{"total":96636764160,"available":96028487680,"internally_reserved":0,"allocated":275853312,"data_stored":275517228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328900608,"data_stored":328897129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275066880,"data_stored":275014102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294985728,"data_stored":294978177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:21.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:21.955 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:21.956 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:22.034 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-15T21:24:20.460831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357143188,"num_objects":87197,"num_object_clones":0,"num_object_copies":174394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180586,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13979,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818888,"kb_used_data":863116,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281296632,"statfs":{"total":289910292480,"available":288047751168,"internally_reserved":0,"allocated":883830784,"data_stored":882917058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":978635786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,3,8,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28114944,"num_objects":-6864,"num_object_clones":0,"num_object_copies":-13728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-443,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6864,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":443,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008125"},"pg_stats":[{"pgid":"2.f","version":"50'13002","reported_seq":13760,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.808008+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:18.808008+0000","last_peered":"2024-09-15T21:24:18.808008+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:18.808008+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13002,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13147","reported_seq":13252,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.553781+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:19.553781+0000","last_peered":"2024-09-15T21:24:19.553781+0000","last_clean":"2024-09-15T21:24:19.553781+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:19.553781+0000","last_undegraded":"2024-09-15T21:24:19.553781+0000","last_fullsized":"2024-09-15T21:24:19.553781+0000","mapping_epoch":36,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22499328,"num_objects":5493,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13211","reported_seq":20135,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.231744+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:19.231744+0000","last_peered":"2024-09-15T21:24:19.231744+0000","last_clean":"2024-09-15T21:24:19.231744+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:19.231744+0000","last_undegraded":"2024-09-15T21:24:19.231744+0000","last_fullsized":"2024-09-15T21:24:19.231744+0000","mapping_epoch":45,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175192,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13218","reported_seq":13294,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.629364+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:19.629364+0000","last_peered":"2024-09-15T21:24:19.629364+0000","last_clean":"2024-09-15T21:24:19.629364+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:19.629364+0000","last_undegraded":"2024-09-15T21:24:19.629364+0000","last_fullsized":"2024-09-15T21:24:19.629364+0000","mapping_epoch":36,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21921792,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13218,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13191","reported_seq":13319,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.848129+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:18.848129+0000","last_peered":"2024-09-15T21:24:18.848129+0000","last_clean":"2024-09-15T21:24:18.848129+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:18.848129+0000","last_undegraded":"2024-09-15T21:24:18.848129+0000","last_fullsized":"2024-09-15T21:24:18.848129+0000","mapping_epoch":35,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22659072,"num_objects":5532,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12985","reported_seq":13805,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.812000+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:18.812000+0000","last_peered":"2024-09-15T21:24:18.812000+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:18.812000+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22102016,"num_objects":5396,"num_object_clones":0,"num_object_copies":10792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10410,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13330","reported_seq":14076,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.893993+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:18.893993+0000","last_peered":"2024-09-15T21:24:18.893993+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:18.893993+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22831104,"num_objects":5574,"num_object_clones":0,"num_object_copies":11148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13037","reported_seq":13166,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.867242+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:18.867242+0000","last_peered":"2024-09-15T21:24:18.867242+0000","last_clean":"2024-09-15T21:24:18.867242+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:18.867242+0000","last_undegraded":"2024-09-15T21:24:18.867242+0000","last_fullsized":"2024-09-15T21:24:18.867242+0000","mapping_epoch":35,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10822,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12950","reported_seq":19773,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.107185+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:19.107185+0000","last_peered":"2024-09-15T21:24:19.107185+0000","last_clean":"2024-09-15T21:24:19.107185+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:19.107185+0000","last_undegraded":"2024-09-15T21:24:19.107185+0000","last_fullsized":"2024-09-15T21:24:19.107185+0000","mapping_epoch":49,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22220800,"num_objects":5425,"num_object_clones":0,"num_object_copies":10850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11536,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12796","reported_seq":13577,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.873603+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:18.873603+0000","last_peered":"2024-09-15T21:24:18.873603+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:18.873603+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21454848,"num_objects":5238,"num_object_clones":0,"num_object_copies":10476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12893","reported_seq":13320,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.525814+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:19.525814+0000","last_peered":"2024-09-15T21:24:19.525814+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:19.525814+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"40'2800","ondisk_log_start":"40'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2280}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13043","reported_seq":13455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.606727+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:19.606727+0000","last_peered":"2024-09-15T21:24:19.606727+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:19.606727+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11442,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2299}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13192","reported_seq":13934,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.822508+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:18.822508+0000","last_peered":"2024-09-15T21:24:18.822508+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:18.822508+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10499,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13032","reported_seq":16366,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.880721+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:18.880721+0000","last_peered":"2024-09-15T21:24:18.880721+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:18.880721+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036480,"num_objects":5380,"num_object_clones":0,"num_object_copies":10760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10617,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1597,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12942","reported_seq":13601,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.880669+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:18.880669+0000","last_peered":"2024-09-15T21:24:18.880669+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:18.880669+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22265856,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10589,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13201","reported_seq":20295,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.192226+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:19.192226+0000","last_peered":"2024-09-15T21:24:19.192226+0000","last_clean":"2024-09-15T21:24:19.192226+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:19.192226+0000","last_undegraded":"2024-09-15T21:24:19.192226+0000","last_fullsized":"2024-09-15T21:24:19.192226+0000","mapping_epoch":47,"log_start":"40'3200","ondisk_log_start":"40'3200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22188649,"num_objects":5418,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10861,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357143169,"num_objects":87195,"num_object_clones":0,"num_object_copies":174390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180584,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13978,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880242688,"data_stored":880178880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554680,"kb_used_data":284188,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93817160,"statfs":{"total":96636764160,"available":96068771840,"internally_reserved":0,"allocated":291008512,"data_stored":290717655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149785,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672608,"kb_used_data":316112,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93699232,"statfs":{"total":96636764160,"available":95948013568,"internally_reserved":0,"allocated":323698688,"data_stored":323411903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,2,0,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626503,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591600,"kb_used_data":262816,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93780240,"statfs":{"total":96636764160,"available":96030965760,"internally_reserved":0,"allocated":269123584,"data_stored":268787500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322039808,"data_stored":322036329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267943936,"data_stored":267891158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290258944,"data_stored":290251393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:22.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:22.815 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:22.897 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-15T21:24:22.461303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357143188,"num_objects":87197,"num_object_clones":0,"num_object_copies":174394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180586,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13979,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818888,"kb_used_data":863116,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281296632,"statfs":{"total":289910292480,"available":288047751168,"internally_reserved":0,"allocated":883830784,"data_stored":882917058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":978635786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,3,8,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18305024,"num_objects":-4469,"num_object_clones":0,"num_object_copies":-8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008173"},"pg_stats":[{"pgid":"2.f","version":"50'13002","reported_seq":13760,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.808008+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:18.808008+0000","last_peered":"2024-09-15T21:24:18.808008+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:18.808008+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13002,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13147","reported_seq":13252,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.553781+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:19.553781+0000","last_peered":"2024-09-15T21:24:19.553781+0000","last_clean":"2024-09-15T21:24:19.553781+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:19.553781+0000","last_undegraded":"2024-09-15T21:24:19.553781+0000","last_fullsized":"2024-09-15T21:24:19.553781+0000","mapping_epoch":36,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22499328,"num_objects":5493,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13211","reported_seq":20135,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.231744+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:19.231744+0000","last_peered":"2024-09-15T21:24:19.231744+0000","last_clean":"2024-09-15T21:24:19.231744+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:19.231744+0000","last_undegraded":"2024-09-15T21:24:19.231744+0000","last_fullsized":"2024-09-15T21:24:19.231744+0000","mapping_epoch":45,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175192,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13218","reported_seq":13294,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.629364+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:19.629364+0000","last_peered":"2024-09-15T21:24:19.629364+0000","last_clean":"2024-09-15T21:24:19.629364+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:19.629364+0000","last_undegraded":"2024-09-15T21:24:19.629364+0000","last_fullsized":"2024-09-15T21:24:19.629364+0000","mapping_epoch":36,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21921792,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13218,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13191","reported_seq":13319,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.848129+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:18.848129+0000","last_peered":"2024-09-15T21:24:18.848129+0000","last_clean":"2024-09-15T21:24:18.848129+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:18.848129+0000","last_undegraded":"2024-09-15T21:24:18.848129+0000","last_fullsized":"2024-09-15T21:24:18.848129+0000","mapping_epoch":35,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22659072,"num_objects":5532,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'12985","reported_seq":13805,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.812000+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:18.812000+0000","last_peered":"2024-09-15T21:24:18.812000+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:18.812000+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22102016,"num_objects":5396,"num_object_clones":0,"num_object_copies":10792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10410,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2367}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13330","reported_seq":14076,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.893993+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:18.893993+0000","last_peered":"2024-09-15T21:24:18.893993+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:18.893993+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22831104,"num_objects":5574,"num_object_clones":0,"num_object_copies":11148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10966,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13037","reported_seq":13166,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.867242+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:18.867242+0000","last_peered":"2024-09-15T21:24:18.867242+0000","last_clean":"2024-09-15T21:24:18.867242+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:18.867242+0000","last_undegraded":"2024-09-15T21:24:18.867242+0000","last_fullsized":"2024-09-15T21:24:18.867242+0000","mapping_epoch":35,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10822,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'12950","reported_seq":19773,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.107185+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:19.107185+0000","last_peered":"2024-09-15T21:24:19.107185+0000","last_clean":"2024-09-15T21:24:19.107185+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:19.107185+0000","last_undegraded":"2024-09-15T21:24:19.107185+0000","last_fullsized":"2024-09-15T21:24:19.107185+0000","mapping_epoch":49,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22220800,"num_objects":5425,"num_object_clones":0,"num_object_copies":10850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11536,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12796","reported_seq":13577,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.873603+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:18.873603+0000","last_peered":"2024-09-15T21:24:18.873603+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:18.873603+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"37'2700","ondisk_log_start":"37'2700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21454848,"num_objects":5238,"num_object_clones":0,"num_object_copies":10476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12893","reported_seq":13320,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.525814+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:19.525814+0000","last_peered":"2024-09-15T21:24:19.525814+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:19.525814+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"40'2800","ondisk_log_start":"40'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2280}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13043","reported_seq":13455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.606727+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:19.606727+0000","last_peered":"2024-09-15T21:24:19.606727+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:19.606727+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11442,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2299}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13192","reported_seq":13934,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.822508+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:18.822508+0000","last_peered":"2024-09-15T21:24:18.822508+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:18.822508+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10499,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2388}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13032","reported_seq":16366,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.880721+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:18.880721+0000","last_peered":"2024-09-15T21:24:18.880721+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:18.880721+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22036480,"num_objects":5380,"num_object_clones":0,"num_object_copies":10760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10617,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1597,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'12942","reported_seq":13601,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.880669+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:18.880669+0000","last_peered":"2024-09-15T21:24:18.880669+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:18.880669+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22265856,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10589,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13201","reported_seq":20295,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.192226+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:19.192226+0000","last_peered":"2024-09-15T21:24:19.192226+0000","last_clean":"2024-09-15T21:24:19.192226+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:19.192226+0000","last_undegraded":"2024-09-15T21:24:19.192226+0000","last_fullsized":"2024-09-15T21:24:19.192226+0000","mapping_epoch":47,"log_start":"40'3200","ondisk_log_start":"40'3200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22188649,"num_objects":5418,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10861,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357143169,"num_objects":87195,"num_object_clones":0,"num_object_copies":174390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180584,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13978,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880242688,"data_stored":880178880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554680,"kb_used_data":284188,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93817160,"statfs":{"total":96636764160,"available":96068771840,"internally_reserved":0,"allocated":291008512,"data_stored":290717655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149785,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672608,"kb_used_data":316112,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93699232,"statfs":{"total":96636764160,"available":95948013568,"internally_reserved":0,"allocated":323698688,"data_stored":323411903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":365031089},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,2,0,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626503,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591600,"kb_used_data":262816,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93780240,"statfs":{"total":96636764160,"available":96030965760,"internally_reserved":0,"allocated":269123584,"data_stored":268787500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322039808,"data_stored":322036329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267943936,"data_stored":267891158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290258944,"data_stored":290251393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13002", 'reported_seq': 13760, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.808008+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:18.808008+0000', 'last_peered': '2024-09-15T21:24:18.808008+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:18.808008+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "40'3000", 'ondisk_log_start': "40'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22568960, 'num_objects': 5510, 'num_object_clones': 0, 'num_object_copies': 11020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13002, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2479}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'12985", 'reported_seq': 13805, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.812000+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:18.812000+0000', 'last_peered': '2024-09-15T21:24:18.812000+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:18.812000+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "40'2900", 'ondisk_log_start': "40'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22102016, 'num_objects': 5396, 'num_object_clones': 0, 'num_object_copies': 10792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5396, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10410, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'13330", 'reported_seq': 14076, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.893993+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:18.893993+0000', 'last_peered': '2024-09-15T21:24:18.893993+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:18.893993+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "41'3300", 'ondisk_log_start': "41'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22831104, 'num_objects': 5574, 'num_object_clones': 0, 'num_object_copies': 11148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10966, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12796", 'reported_seq': 13577, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.873603+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:18.873603+0000', 'last_peered': '2024-09-15T21:24:18.873603+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:18.873603+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "37'2700", 'ondisk_log_start': "37'2700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21454848, 'num_objects': 5238, 'num_object_clones': 0, 'num_object_copies': 10476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10322, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2223}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'12893", 'reported_seq': 13320, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:19.525814+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:19.525814+0000', 'last_peered': '2024-09-15T21:24:19.525814+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:19.525814+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "40'2800", 'ondisk_log_start': "40'2800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22085632, 'num_objects': 5392, 'num_object_clones': 0, 'num_object_copies': 10784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11444, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2280}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13043", 'reported_seq': 13455, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:19.606727+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:19.606727+0000', 'last_peered': '2024-09-15T21:24:19.606727+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:19.606727+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "40'3000", 'ondisk_log_start': "40'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22548480, 'num_objects': 5505, 'num_object_clones': 0, 'num_object_copies': 11010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11442, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2299}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13192", 'reported_seq': 13934, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.822508+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:18.822508+0000', 'last_peered': '2024-09-15T21:24:18.822508+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:18.822508+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "40'3100", 'ondisk_log_start': "40'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22601728, 'num_objects': 5518, 'num_object_clones': 0, 'num_object_copies': 11036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10499, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'13032", 'reported_seq': 16366, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.880721+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:18.880721+0000', 'last_peered': '2024-09-15T21:24:18.880721+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:18.880721+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "40'3000", 'ondisk_log_start': "40'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22036480, 'num_objects': 5380, 'num_object_clones': 0, 'num_object_copies': 10760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10617, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1597, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:22.900 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'12942", 'reported_seq': 13601, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.880669+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:18.880669+0000', 'last_peered': '2024-09-15T21:24:18.880669+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:18.880669+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "40'2900", 'ondisk_log_start': "40'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22265856, 'num_objects': 5436, 'num_object_clones': 0, 'num_object_copies': 10872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10589, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2334}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:22.901 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:23.961 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:23.962 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26587/174394 objects degraded (15.245%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87197,"data_bytes":357143188,"bytes_used":1862541312,"bytes_avail":288047751168,"bytes_total":289910292480,"degraded_objects":26587,"degraded_total":174394,"degraded_ratio":0.15245363946007318,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-15T21:24:20.460939+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:24.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:24.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:24.965 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:24.966 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:25.061 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-15T21:24:24.461718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":349840020,"num_objects":85414,"num_object_clones":0,"num_object_copies":170828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182369,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14134,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804516,"kb_used_data":848740,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281311004,"statfs":{"total":289910292480,"available":288062468096,"internally_reserved":0,"allocated":869109760,"data_stored":868196034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":978635786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,6,0,0,8,5,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25608192,"num_objects":-6252,"num_object_clones":0,"num_object_copies":-12504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-433,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6252,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":433,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008101"},"pg_stats":[{"pgid":"2.f","version":"50'13168","reported_seq":13926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.907325+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:23.907325+0000","last_peered":"2024-09-15T21:24:23.907325+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:23.907325+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13168,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13147","reported_seq":13252,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.553781+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:19.553781+0000","last_peered":"2024-09-15T21:24:19.553781+0000","last_clean":"2024-09-15T21:24:19.553781+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:19.553781+0000","last_undegraded":"2024-09-15T21:24:19.553781+0000","last_fullsized":"2024-09-15T21:24:19.553781+0000","mapping_epoch":36,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22499328,"num_objects":5493,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11873,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13336","reported_seq":20261,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.185731+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:24.185731+0000","last_peered":"2024-09-15T21:24:24.185731+0000","last_clean":"2024-09-15T21:24:24.185731+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:24.185731+0000","last_undegraded":"2024-09-15T21:24:24.185731+0000","last_fullsized":"2024-09-15T21:24:24.185731+0000","mapping_epoch":45,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663192,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12059,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13218","reported_seq":13294,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:19.629364+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:19.629364+0000","last_peered":"2024-09-15T21:24:19.629364+0000","last_clean":"2024-09-15T21:24:19.629364+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:19.629364+0000","last_undegraded":"2024-09-15T21:24:19.629364+0000","last_fullsized":"2024-09-15T21:24:19.629364+0000","mapping_epoch":36,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21921792,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13218,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13346","reported_seq":13474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.886237+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:23.886237+0000","last_peered":"2024-09-15T21:24:23.886237+0000","last_clean":"2024-09-15T21:24:23.886237+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:23.886237+0000","last_undegraded":"2024-09-15T21:24:23.886237+0000","last_fullsized":"2024-09-15T21:24:23.886237+0000","mapping_epoch":35,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11204,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13133","reported_seq":13956,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863741+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:23.863741+0000","last_peered":"2024-09-15T21:24:23.863741+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:23.863741+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21495808,"num_objects":5248,"num_object_clones":0,"num_object_copies":10496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2219}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13463","reported_seq":14205,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.841223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:23.841223+0000","last_peered":"2024-09-15T21:24:23.841223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:23.841223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22286336,"num_objects":5441,"num_object_clones":0,"num_object_copies":10882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11099,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13183","reported_seq":13312,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.857167+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:23.857167+0000","last_peered":"2024-09-15T21:24:23.857167+0000","last_clean":"2024-09-15T21:24:23.857167+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:23.857167+0000","last_undegraded":"2024-09-15T21:24:23.857167+0000","last_fullsized":"2024-09-15T21:24:23.857167+0000","mapping_epoch":35,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385216,"num_objects":5221,"num_object_clones":0,"num_object_copies":10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10968,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13088","reported_seq":19911,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.231933+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:24.231933+0000","last_peered":"2024-09-15T21:24:24.231933+0000","last_clean":"2024-09-15T21:24:24.231933+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:24.231933+0000","last_undegraded":"2024-09-15T21:24:24.231933+0000","last_fullsized":"2024-09-15T21:24:24.231933+0000","mapping_epoch":49,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11674,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12938","reported_seq":13722,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:23.863949+0000","last_peered":"2024-09-15T21:24:23.863949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:23.863949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20873216,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'12893","reported_seq":13320,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.525814+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:19.525814+0000","last_peered":"2024-09-15T21:24:19.525814+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:19.525814+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"40'2800","ondisk_log_start":"40'2800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2280}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13043","reported_seq":13455,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:19.606727+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:19.606727+0000","last_peered":"2024-09-15T21:24:19.606727+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:19.606727+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11442,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2299}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13354","reported_seq":14095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.861030+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:23.861030+0000","last_peered":"2024-09-15T21:24:23.861030+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:23.861030+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10661,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13179","reported_seq":16823,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.898666+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:23.898666+0000","last_peered":"2024-09-15T21:24:23.898666+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:23.898666+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10764,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1752,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13101","reported_seq":13760,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.807639+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:23.807639+0000","last_peered":"2024-09-15T21:24:23.807639+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:23.807639+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21614592,"num_objects":5277,"num_object_clones":0,"num_object_copies":10554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13363","reported_seq":20457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.045345+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:24.045345+0000","last_peered":"2024-09-15T21:24:24.045345+0000","last_clean":"2024-09-15T21:24:24.045345+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:24.045345+0000","last_undegraded":"2024-09-15T21:24:24.045345+0000","last_fullsized":"2024-09-15T21:24:24.045345+0000","mapping_epoch":47,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21525097,"num_objects":5256,"num_object_clones":0,"num_object_copies":10512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":349840001,"num_objects":85412,"num_object_clones":0,"num_object_copies":170824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182367,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14133,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":865538048,"data_stored":865474240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554680,"kb_used_data":284188,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93817160,"statfs":{"total":96636764160,"available":96068771840,"internally_reserved":0,"allocated":291008512,"data_stored":290717655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673588,"kb_used_data":308772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93698252,"statfs":{"total":96636764160,"available":95947010048,"internally_reserved":0,"allocated":316182528,"data_stored":315895743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":373550769},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,0,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576248,"kb_used_data":255780,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93795592,"statfs":{"total":96636764160,"available":96046686208,"internally_reserved":0,"allocated":261918720,"data_stored":261582636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314646528,"data_stored":314643049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260632576,"data_stored":260579798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290258944,"data_stored":290251393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:28.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:28.935 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:29.029 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-15T21:24:28.462530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":347558548,"num_objects":84857,"num_object_clones":0,"num_object_copies":169714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182926,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14134,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804100,"kb_used_data":844176,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":281311420,"statfs":{"total":289910292480,"available":288062894080,"internally_reserved":0,"allocated":864436224,"data_stored":863522498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":982895626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,6,0,0,9,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18886656,"num_objects":-4611,"num_object_clones":0,"num_object_copies":-9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004076"},"pg_stats":[{"pgid":"2.f","version":"50'13168","reported_seq":13926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.907325+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:23.907325+0000","last_peered":"2024-09-15T21:24:23.907325+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:23.907325+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13168,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13295","reported_seq":13400,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.708045+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:24.708045+0000","last_peered":"2024-09-15T21:24:24.708045+0000","last_clean":"2024-09-15T21:24:24.708045+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:24.708045+0000","last_undegraded":"2024-09-15T21:24:24.708045+0000","last_fullsized":"2024-09-15T21:24:24.708045+0000","mapping_epoch":36,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13336","reported_seq":20261,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.185731+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:24.185731+0000","last_peered":"2024-09-15T21:24:24.185731+0000","last_clean":"2024-09-15T21:24:24.185731+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:24.185731+0000","last_undegraded":"2024-09-15T21:24:24.185731+0000","last_fullsized":"2024-09-15T21:24:24.185731+0000","mapping_epoch":45,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663192,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12059,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13334","reported_seq":13413,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.649381+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:24.649381+0000","last_peered":"2024-09-15T21:24:24.649381+0000","last_clean":"2024-09-15T21:24:24.649381+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:24.649381+0000","last_undegraded":"2024-09-15T21:24:24.649381+0000","last_fullsized":"2024-09-15T21:24:24.649381+0000","mapping_epoch":36,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13346","reported_seq":13474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.886237+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:23.886237+0000","last_peered":"2024-09-15T21:24:23.886237+0000","last_clean":"2024-09-15T21:24:23.886237+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:23.886237+0000","last_undegraded":"2024-09-15T21:24:23.886237+0000","last_fullsized":"2024-09-15T21:24:23.886237+0000","mapping_epoch":35,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11204,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13133","reported_seq":13956,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863741+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:23.863741+0000","last_peered":"2024-09-15T21:24:23.863741+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:23.863741+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21495808,"num_objects":5248,"num_object_clones":0,"num_object_copies":10496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2219}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13463","reported_seq":14205,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.841223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:23.841223+0000","last_peered":"2024-09-15T21:24:23.841223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:23.841223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22286336,"num_objects":5441,"num_object_clones":0,"num_object_copies":10882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11099,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13183","reported_seq":13312,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.857167+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:23.857167+0000","last_peered":"2024-09-15T21:24:23.857167+0000","last_clean":"2024-09-15T21:24:23.857167+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:23.857167+0000","last_undegraded":"2024-09-15T21:24:23.857167+0000","last_fullsized":"2024-09-15T21:24:23.857167+0000","mapping_epoch":35,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385216,"num_objects":5221,"num_object_clones":0,"num_object_copies":10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10968,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13088","reported_seq":19911,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.231933+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:24.231933+0000","last_peered":"2024-09-15T21:24:24.231933+0000","last_clean":"2024-09-15T21:24:24.231933+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:24.231933+0000","last_undegraded":"2024-09-15T21:24:24.231933+0000","last_fullsized":"2024-09-15T21:24:24.231933+0000","mapping_epoch":49,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11674,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12938","reported_seq":13722,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:23.863949+0000","last_peered":"2024-09-15T21:24:23.863949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:23.863949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20873216,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13026","reported_seq":13453,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:24.605137+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:24.605137+0000","last_peered":"2024-09-15T21:24:24.605137+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:24.605137+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21540864,"num_objects":5259,"num_object_clones":0,"num_object_copies":10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13203","reported_seq":13615,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:24.656028+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:24.656028+0000","last_peered":"2024-09-15T21:24:24.656028+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:24.656028+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2139}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13354","reported_seq":14095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.861030+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:23.861030+0000","last_peered":"2024-09-15T21:24:23.861030+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:23.861030+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10661,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13179","reported_seq":16823,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.898666+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:23.898666+0000","last_peered":"2024-09-15T21:24:23.898666+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:23.898666+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10764,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1752,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13101","reported_seq":13760,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.807639+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:23.807639+0000","last_peered":"2024-09-15T21:24:23.807639+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:23.807639+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21614592,"num_objects":5277,"num_object_clones":0,"num_object_copies":10554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13363","reported_seq":20457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.045345+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:24.045345+0000","last_peered":"2024-09-15T21:24:24.045345+0000","last_clean":"2024-09-15T21:24:24.045345+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:24.045345+0000","last_undegraded":"2024-09-15T21:24:24.045345+0000","last_fullsized":"2024-09-15T21:24:24.045345+0000","mapping_epoch":47,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21525097,"num_objects":5256,"num_object_clones":0,"num_object_copies":10512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":347558529,"num_objects":84855,"num_object_clones":0,"num_object_copies":169710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182924,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14133,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860876800,"data_stored":860812992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953559,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554264,"kb_used_data":279624,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93817576,"statfs":{"total":96636764160,"available":96069197824,"internally_reserved":0,"allocated":286334976,"data_stored":286044119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673588,"kb_used_data":308772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93698252,"statfs":{"total":96636764160,"available":95947010048,"internally_reserved":0,"allocated":316182528,"data_stored":315895743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":373550769},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,0,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576248,"kb_used_data":255780,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93795592,"statfs":{"total":96636764160,"available":96046686208,"internally_reserved":0,"allocated":261918720,"data_stored":261582636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314646528,"data_stored":314643049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260632576,"data_stored":260579798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285597696,"data_stored":285590145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:29.030 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:29.804 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:29.805 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:29.860 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-15T21:24:28.462530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":347558548,"num_objects":84857,"num_object_clones":0,"num_object_copies":169714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182926,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14134,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804100,"kb_used_data":844176,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":281311420,"statfs":{"total":289910292480,"available":288062894080,"internally_reserved":0,"allocated":864436224,"data_stored":863522498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":982895626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,6,0,0,9,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18886656,"num_objects":-4611,"num_object_clones":0,"num_object_copies":-9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004076"},"pg_stats":[{"pgid":"2.f","version":"50'13168","reported_seq":13926,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.907325+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:23.907325+0000","last_peered":"2024-09-15T21:24:23.907325+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:23.907325+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13168,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13295","reported_seq":13400,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.708045+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:24.708045+0000","last_peered":"2024-09-15T21:24:24.708045+0000","last_clean":"2024-09-15T21:24:24.708045+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:24.708045+0000","last_undegraded":"2024-09-15T21:24:24.708045+0000","last_fullsized":"2024-09-15T21:24:24.708045+0000","mapping_epoch":36,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13336","reported_seq":20261,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.185731+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:24.185731+0000","last_peered":"2024-09-15T21:24:24.185731+0000","last_clean":"2024-09-15T21:24:24.185731+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:24.185731+0000","last_undegraded":"2024-09-15T21:24:24.185731+0000","last_fullsized":"2024-09-15T21:24:24.185731+0000","mapping_epoch":45,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663192,"num_objects":5534,"num_object_clones":0,"num_object_copies":11068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12059,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13334","reported_seq":13413,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.649381+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:24.649381+0000","last_peered":"2024-09-15T21:24:24.649381+0000","last_clean":"2024-09-15T21:24:24.649381+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:24.649381+0000","last_undegraded":"2024-09-15T21:24:24.649381+0000","last_fullsized":"2024-09-15T21:24:24.649381+0000","mapping_epoch":36,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13346","reported_seq":13474,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.886237+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:23.886237+0000","last_peered":"2024-09-15T21:24:23.886237+0000","last_clean":"2024-09-15T21:24:23.886237+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:23.886237+0000","last_undegraded":"2024-09-15T21:24:23.886237+0000","last_fullsized":"2024-09-15T21:24:23.886237+0000","mapping_epoch":35,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11204,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13133","reported_seq":13956,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863741+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:23.863741+0000","last_peered":"2024-09-15T21:24:23.863741+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:23.863741+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21495808,"num_objects":5248,"num_object_clones":0,"num_object_copies":10496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2219}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13463","reported_seq":14205,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.841223+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:23.841223+0000","last_peered":"2024-09-15T21:24:23.841223+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:23.841223+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22286336,"num_objects":5441,"num_object_clones":0,"num_object_copies":10882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11099,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13183","reported_seq":13312,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.857167+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:23.857167+0000","last_peered":"2024-09-15T21:24:23.857167+0000","last_clean":"2024-09-15T21:24:23.857167+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:23.857167+0000","last_undegraded":"2024-09-15T21:24:23.857167+0000","last_fullsized":"2024-09-15T21:24:23.857167+0000","mapping_epoch":35,"log_start":"40'3100","ondisk_log_start":"40'3100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385216,"num_objects":5221,"num_object_clones":0,"num_object_copies":10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10968,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13088","reported_seq":19911,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.231933+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:24.231933+0000","last_peered":"2024-09-15T21:24:24.231933+0000","last_clean":"2024-09-15T21:24:24.231933+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:24.231933+0000","last_undegraded":"2024-09-15T21:24:24.231933+0000","last_fullsized":"2024-09-15T21:24:24.231933+0000","mapping_epoch":49,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11674,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'12938","reported_seq":13722,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.863949+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:23.863949+0000","last_peered":"2024-09-15T21:24:23.863949+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:23.863949+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"40'2900","ondisk_log_start":"40'2900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20873216,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13026","reported_seq":13453,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:24.605137+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:24.605137+0000","last_peered":"2024-09-15T21:24:24.605137+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:24.605137+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21540864,"num_objects":5259,"num_object_clones":0,"num_object_copies":10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11577,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13203","reported_seq":13615,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:24.656028+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:24.656028+0000","last_peered":"2024-09-15T21:24:24.656028+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:24.656028+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11602,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2139}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13354","reported_seq":14095,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.861030+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:23.861030+0000","last_peered":"2024-09-15T21:24:23.861030+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:23.861030+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21938176,"num_objects":5356,"num_object_clones":0,"num_object_copies":10712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10661,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13179","reported_seq":16823,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.898666+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:23.898666+0000","last_peered":"2024-09-15T21:24:23.898666+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:23.898666+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21434368,"num_objects":5233,"num_object_clones":0,"num_object_copies":10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10764,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1752,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13101","reported_seq":13760,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.807639+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:23.807639+0000","last_peered":"2024-09-15T21:24:23.807639+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:23.807639+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21614592,"num_objects":5277,"num_object_clones":0,"num_object_copies":10554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13363","reported_seq":20457,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:24.045345+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:24.045345+0000","last_peered":"2024-09-15T21:24:24.045345+0000","last_clean":"2024-09-15T21:24:24.045345+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:24.045345+0000","last_undegraded":"2024-09-15T21:24:24.045345+0000","last_fullsized":"2024-09-15T21:24:24.045345+0000","mapping_epoch":47,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21525097,"num_objects":5256,"num_object_clones":0,"num_object_copies":10512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":347558529,"num_objects":84855,"num_object_clones":0,"num_object_copies":169710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182924,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14133,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860876800,"data_stored":860812992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953559,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554264,"kb_used_data":279624,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93817576,"statfs":{"total":96636764160,"available":96069197824,"internally_reserved":0,"allocated":286334976,"data_stored":286044119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673588,"kb_used_data":308772,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93698252,"statfs":{"total":96636764160,"available":95947010048,"internally_reserved":0,"allocated":316182528,"data_stored":315895743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":373550769},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,0,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626504,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576248,"kb_used_data":255780,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93795592,"statfs":{"total":96636764160,"available":96046686208,"internally_reserved":0,"allocated":261918720,"data_stored":261582636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,0,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314646528,"data_stored":314643049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260632576,"data_stored":260579798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":285597696,"data_stored":285590145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13168", 'reported_seq': 13926, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.907325+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:23.907325+0000', 'last_peered': '2024-09-15T21:24:23.907325+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:23.907325+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "41'3100", 'ondisk_log_start': "41'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21889024, 'num_objects': 5344, 'num_object_clones': 0, 'num_object_copies': 10688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13168, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2313}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'13133", 'reported_seq': 13956, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.863741+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:23.863741+0000', 'last_peered': '2024-09-15T21:24:23.863741+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:23.863741+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "40'3100", 'ondisk_log_start': "40'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21495808, 'num_objects': 5248, 'num_object_clones': 0, 'num_object_copies': 10496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10558, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2219}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'13463", 'reported_seq': 14205, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.841223+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:23.841223+0000', 'last_peered': '2024-09-15T21:24:23.841223+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:23.841223+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "41'3400", 'ondisk_log_start': "41'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22286336, 'num_objects': 5441, 'num_object_clones': 0, 'num_object_copies': 10882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5441, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11099, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2277}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'12938", 'reported_seq': 13722, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.863949+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:23.863949+0000', 'last_peered': '2024-09-15T21:24:23.863949+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:23.863949+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "40'2900", 'ondisk_log_start': "40'2900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20873216, 'num_objects': 5096, 'num_object_clones': 0, 'num_object_copies': 10192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10464, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2081}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'13026", 'reported_seq': 13453, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:24.605137+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:24.605137+0000', 'last_peered': '2024-09-15T21:24:24.605137+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:24.605137+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "40'3000", 'ondisk_log_start': "40'3000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21540864, 'num_objects': 5259, 'num_object_clones': 0, 'num_object_copies': 10518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11577, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13203", 'reported_seq': 13615, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:24.656028+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:24.656028+0000', 'last_peered': '2024-09-15T21:24:24.656028+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:24.656028+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "41'3200", 'ondisk_log_start': "41'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21893120, 'num_objects': 5345, 'num_object_clones': 0, 'num_object_copies': 10690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11602, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2139}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13354", 'reported_seq': 14095, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.861030+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:23.861030+0000', 'last_peered': '2024-09-15T21:24:23.861030+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:23.861030+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "41'3300", 'ondisk_log_start': "41'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21938176, 'num_objects': 5356, 'num_object_clones': 0, 'num_object_copies': 10712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10661, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'13179", 'reported_seq': 16823, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.898666+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:23.898666+0000', 'last_peered': '2024-09-15T21:24:23.898666+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:23.898666+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "41'3100", 'ondisk_log_start': "41'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21434368, 'num_objects': 5233, 'num_object_clones': 0, 'num_object_copies': 10466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1630, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10764, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1752, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:29.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'13101", 'reported_seq': 13760, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:23.807639+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:23.807639+0000', 'last_peered': '2024-09-15T21:24:23.807639+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:23.807639+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "41'3100", 'ondisk_log_start': "41'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21614592, 'num_objects': 5277, 'num_object_clones': 0, 'num_object_copies': 10554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10748, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.864 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:30.874 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:30.874 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26262/165074 objects degraded (15.909%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":483,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82537,"data_bytes":338055828,"bytes_used":1830125568,"bytes_avail":288080166912,"bytes_total":289910292480,"degraded_objects":26262,"degraded_total":165074,"degraded_ratio":0.15909228588390661,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":449,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-15T21:24:20.460939+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:30.931 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:30.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:31.786 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:31.788 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:31.867 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-15T21:24:30.462956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338055828,"num_objects":82537,"num_object_clones":0,"num_object_copies":165074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185246,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14304,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160633,"ondisk_log_size":160633,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787232,"kb_used_data":825876,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281328288,"statfs":{"total":289910292480,"available":288080166912,"internally_reserved":0,"allocated":845697024,"data_stored":844783298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":984337418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,13,10,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28389376,"num_objects":-6931,"num_object_clones":0,"num_object_copies":-13862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-449,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002536"},"pg_stats":[{"pgid":"2.f","version":"50'13311","reported_seq":14069,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.929007+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:28.929007+0000","last_peered":"2024-09-15T21:24:28.929007+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:28.929007+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13460","reported_seq":13566,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.677321+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:29.677321+0000","last_peered":"2024-09-15T21:24:29.677321+0000","last_clean":"2024-09-15T21:24:29.677321+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:29.677321+0000","last_undegraded":"2024-09-15T21:24:29.677321+0000","last_fullsized":"2024-09-15T21:24:29.677321+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13508","reported_seq":20433,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.273960+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:29.273960+0000","last_peered":"2024-09-15T21:24:29.273960+0000","last_clean":"2024-09-15T21:24:29.273960+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:29.273960+0000","last_undegraded":"2024-09-15T21:24:29.273960+0000","last_fullsized":"2024-09-15T21:24:29.273960+0000","mapping_epoch":45,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21958680,"num_objects":5362,"num_object_clones":0,"num_object_copies":10724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12231,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13471","reported_seq":13550,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.588690+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:29.588690+0000","last_peered":"2024-09-15T21:24:29.588690+0000","last_clean":"2024-09-15T21:24:29.588690+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:29.588690+0000","last_undegraded":"2024-09-15T21:24:29.588690+0000","last_fullsized":"2024-09-15T21:24:29.588690+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20885504,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13471,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13501","reported_seq":13629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.777470+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:28.777470+0000","last_peered":"2024-09-15T21:24:28.777470+0000","last_clean":"2024-09-15T21:24:28.777470+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:28.777470+0000","last_undegraded":"2024-09-15T21:24:28.777470+0000","last_fullsized":"2024-09-15T21:24:28.777470+0000","mapping_epoch":35,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21389312,"num_objects":5222,"num_object_clones":0,"num_object_copies":10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11359,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13271","reported_seq":14094,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.873727+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:28.873727+0000","last_peered":"2024-09-15T21:24:28.873727+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:28.873727+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20930560,"num_objects":5110,"num_object_clones":0,"num_object_copies":10220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10696,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2081}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13608","reported_seq":14354,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.884503+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:28.884503+0000","last_peered":"2024-09-15T21:24:28.884503+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:28.884503+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11244,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13325","reported_seq":13454,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.840622+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:28.840622+0000","last_peered":"2024-09-15T21:24:28.840622+0000","last_clean":"2024-09-15T21:24:28.840622+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:28.840622+0000","last_undegraded":"2024-09-15T21:24:28.840622+0000","last_fullsized":"2024-09-15T21:24:28.840622+0000","mapping_epoch":35,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20803584,"num_objects":5079,"num_object_clones":0,"num_object_copies":10158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11110,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13238","reported_seq":20061,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.323643+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:29.323643+0000","last_peered":"2024-09-15T21:24:29.323643+0000","last_clean":"2024-09-15T21:24:29.323643+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:29.323643+0000","last_undegraded":"2024-09-15T21:24:29.323643+0000","last_fullsized":"2024-09-15T21:24:29.323643+0000","mapping_epoch":49,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21041152,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11824,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13061","reported_seq":13845,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.761626+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:28.761626+0000","last_peered":"2024-09-15T21:24:28.761626+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:28.761626+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"40'3000","ondisk_log_start":"40'3000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369408,"num_objects":4973,"num_object_clones":0,"num_object_copies":9946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10587,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1958}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13157","reported_seq":13582,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.684238+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:29.684238+0000","last_peered":"2024-09-15T21:24:29.684238+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:29.684238+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11708,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2016}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13352","reported_seq":13764,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.709963+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:29.709963+0000","last_peered":"2024-09-15T21:24:29.709963+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:29.709963+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13518","reported_seq":14260,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.862792+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:28.862792+0000","last_peered":"2024-09-15T21:24:28.862792+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:28.862792+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21266432,"num_objects":5192,"num_object_clones":0,"num_object_copies":10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10825,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2062}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13314","reported_seq":17298,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.925327+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:28.925327+0000","last_peered":"2024-09-15T21:24:28.925327+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:28.925327+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1922,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13242","reported_seq":13900,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.880445+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:28.880445+0000","last_peered":"2024-09-15T21:24:28.880445+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:28.880445+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037056,"num_objects":5136,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10889,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2034}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13493","reported_seq":20585,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.356690+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:29.356690+0000","last_peered":"2024-09-15T21:24:29.356690+0000","last_clean":"2024-09-15T21:24:29.356690+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:29.356690+0000","last_undegraded":"2024-09-15T21:24:29.356690+0000","last_fullsized":"2024-09-15T21:24:29.356690+0000","mapping_epoch":47,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20992617,"num_objects":5126,"num_object_clones":0,"num_object_copies":10252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11153,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338055809,"num_objects":82535,"num_object_clones":0,"num_object_copies":165070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185244,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14303,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841879552,"data_stored":841815744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160630,"ondisk_log_size":160630,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":274856,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":281452544,"data_stored":281161687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149787,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":302052,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":309301248,"data_stored":309014463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":379252401},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577776,"kb_used_data":248968,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93794064,"statfs":{"total":96636764160,"available":96045121536,"internally_reserved":0,"allocated":254943232,"data_stored":254607148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,2,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307658752,"data_stored":307655273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253534208,"data_stored":253481430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280686592,"data_stored":280679041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:34.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:35.526 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:35.527 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:35.543 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-15T21:24:34.463921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330625684,"num_objects":80723,"num_object_clones":0,"num_object_copies":161446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187060,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14452,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768428,"kb_used_data":811228,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":281347092,"statfs":{"total":289910292480,"available":288099422208,"internally_reserved":0,"allocated":830697472,"data_stored":829783746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":980077578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,14,7,5,17,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26517504,"num_objects":-6474,"num_object_clones":0,"num_object_copies":-12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002618"},"pg_stats":[{"pgid":"2.f","version":"50'13452","reported_seq":14210,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.910197+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:33.910197+0000","last_peered":"2024-09-15T21:24:33.910197+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:33.910197+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725760,"num_objects":5060,"num_object_clones":0,"num_object_copies":10120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13452,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13460","reported_seq":13566,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.677321+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:29.677321+0000","last_peered":"2024-09-15T21:24:29.677321+0000","last_clean":"2024-09-15T21:24:29.677321+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:29.677321+0000","last_undegraded":"2024-09-15T21:24:29.677321+0000","last_fullsized":"2024-09-15T21:24:29.677321+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13648","reported_seq":20573,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.350678+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:34.350678+0000","last_peered":"2024-09-15T21:24:34.350678+0000","last_clean":"2024-09-15T21:24:34.350678+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:34.350678+0000","last_undegraded":"2024-09-15T21:24:34.350678+0000","last_fullsized":"2024-09-15T21:24:34.350678+0000","mapping_epoch":45,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385240,"num_objects":5222,"num_object_clones":0,"num_object_copies":10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12371,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13471","reported_seq":13550,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.588690+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:29.588690+0000","last_peered":"2024-09-15T21:24:29.588690+0000","last_clean":"2024-09-15T21:24:29.588690+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:29.588690+0000","last_undegraded":"2024-09-15T21:24:29.588690+0000","last_fullsized":"2024-09-15T21:24:29.588690+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20885504,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13471,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13640","reported_seq":13768,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.881400+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:33.881400+0000","last_peered":"2024-09-15T21:24:33.881400+0000","last_clean":"2024-09-15T21:24:33.881400+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:33.881400+0000","last_undegraded":"2024-09-15T21:24:33.881400+0000","last_fullsized":"2024-09-15T21:24:33.881400+0000","mapping_epoch":35,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20819968,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13437","reported_seq":14257,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822733+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:33.822733+0000","last_peered":"2024-09-15T21:24:33.822733+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:33.822733+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13772","reported_seq":14518,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.909425+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:33.909425+0000","last_peered":"2024-09-15T21:24:33.909425+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:33.909425+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21020672,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13469","reported_seq":13598,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.900349+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:33.900349+0000","last_peered":"2024-09-15T21:24:33.900349+0000","last_clean":"2024-09-15T21:24:33.900349+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:33.900349+0000","last_undegraded":"2024-09-15T21:24:33.900349+0000","last_fullsized":"2024-09-15T21:24:33.900349+0000","mapping_epoch":35,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11254,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13402","reported_seq":20225,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.363454+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:34.363454+0000","last_peered":"2024-09-15T21:24:34.363454+0000","last_clean":"2024-09-15T21:24:34.363454+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:34.363454+0000","last_undegraded":"2024-09-15T21:24:34.363454+0000","last_fullsized":"2024-09-15T21:24:34.363454+0000","mapping_epoch":49,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369408,"num_objects":4973,"num_object_clones":0,"num_object_copies":9946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11988,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13221","reported_seq":14005,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.821954+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:33.821954+0000","last_peered":"2024-09-15T21:24:33.821954+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:33.821954+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10747,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13157","reported_seq":13582,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.684238+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:29.684238+0000","last_peered":"2024-09-15T21:24:29.684238+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:29.684238+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11708,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2016}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13352","reported_seq":13764,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.709963+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:29.709963+0000","last_peered":"2024-09-15T21:24:29.709963+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:29.709963+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13663","reported_seq":14405,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822825+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:33.822825+0000","last_peered":"2024-09-15T21:24:33.822825+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:33.822825+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10970,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13446","reported_seq":17726,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.948771+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:33.948771+0000","last_peered":"2024-09-15T21:24:33.948771+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:33.948771+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13402","reported_seq":14061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.853331+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:33.853331+0000","last_peered":"2024-09-15T21:24:33.853331+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:33.853331+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13652","reported_seq":20746,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.302730+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:34.302730+0000","last_peered":"2024-09-15T21:24:34.302730+0000","last_clean":"2024-09-15T21:24:34.302730+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:34.302730+0000","last_undegraded":"2024-09-15T21:24:34.302730+0000","last_fullsized":"2024-09-15T21:24:34.302730+0000","mapping_epoch":47,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20341353,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330625665,"num_objects":80721,"num_object_clones":0,"num_object_copies":161442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80721,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187058,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14451,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":826867712,"data_stored":826803904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160744,"ondisk_log_size":160744,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":274856,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":281452544,"data_stored":281161687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673556,"kb_used_data":294832,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93698284,"statfs":{"total":96636764160,"available":95947042816,"internally_reserved":0,"allocated":301907968,"data_stored":301621183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,3,0,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557856,"kb_used_data":241540,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93813984,"statfs":{"total":96636764160,"available":96065519616,"internally_reserved":0,"allocated":247336960,"data_stored":247000876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300183552,"data_stored":300180073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245997568,"data_stored":245944790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280686592,"data_stored":280679041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:35.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:36.363 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:36.414 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-15T21:24:34.463921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330625684,"num_objects":80723,"num_object_clones":0,"num_object_copies":161446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187060,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14452,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768428,"kb_used_data":811228,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":281347092,"statfs":{"total":289910292480,"available":288099422208,"internally_reserved":0,"allocated":830697472,"data_stored":829783746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":980077578},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,14,7,5,17,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26517504,"num_objects":-6474,"num_object_clones":0,"num_object_copies":-12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6474,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002618"},"pg_stats":[{"pgid":"2.f","version":"50'13452","reported_seq":14210,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.910197+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:33.910197+0000","last_peered":"2024-09-15T21:24:33.910197+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:33.910197+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725760,"num_objects":5060,"num_object_clones":0,"num_object_copies":10120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13452,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13460","reported_seq":13566,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.677321+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:29.677321+0000","last_peered":"2024-09-15T21:24:29.677321+0000","last_clean":"2024-09-15T21:24:29.677321+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:29.677321+0000","last_undegraded":"2024-09-15T21:24:29.677321+0000","last_fullsized":"2024-09-15T21:24:29.677321+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13648","reported_seq":20573,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.350678+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:34.350678+0000","last_peered":"2024-09-15T21:24:34.350678+0000","last_clean":"2024-09-15T21:24:34.350678+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:34.350678+0000","last_undegraded":"2024-09-15T21:24:34.350678+0000","last_fullsized":"2024-09-15T21:24:34.350678+0000","mapping_epoch":45,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385240,"num_objects":5222,"num_object_clones":0,"num_object_copies":10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12371,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13471","reported_seq":13550,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:29.588690+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:29.588690+0000","last_peered":"2024-09-15T21:24:29.588690+0000","last_clean":"2024-09-15T21:24:29.588690+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:29.588690+0000","last_undegraded":"2024-09-15T21:24:29.588690+0000","last_fullsized":"2024-09-15T21:24:29.588690+0000","mapping_epoch":36,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20885504,"num_objects":5099,"num_object_clones":0,"num_object_copies":10198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13471,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13640","reported_seq":13768,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.881400+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:33.881400+0000","last_peered":"2024-09-15T21:24:33.881400+0000","last_clean":"2024-09-15T21:24:33.881400+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:33.881400+0000","last_undegraded":"2024-09-15T21:24:33.881400+0000","last_fullsized":"2024-09-15T21:24:33.881400+0000","mapping_epoch":35,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20819968,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13437","reported_seq":14257,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822733+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:33.822733+0000","last_peered":"2024-09-15T21:24:33.822733+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:33.822733+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13772","reported_seq":14518,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.909425+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:33.909425+0000","last_peered":"2024-09-15T21:24:33.909425+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:33.909425+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21020672,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13469","reported_seq":13598,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.900349+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:33.900349+0000","last_peered":"2024-09-15T21:24:33.900349+0000","last_clean":"2024-09-15T21:24:33.900349+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:33.900349+0000","last_undegraded":"2024-09-15T21:24:33.900349+0000","last_fullsized":"2024-09-15T21:24:33.900349+0000","mapping_epoch":35,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11254,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13402","reported_seq":20225,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.363454+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:34.363454+0000","last_peered":"2024-09-15T21:24:34.363454+0000","last_clean":"2024-09-15T21:24:34.363454+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:34.363454+0000","last_undegraded":"2024-09-15T21:24:34.363454+0000","last_fullsized":"2024-09-15T21:24:34.363454+0000","mapping_epoch":49,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369408,"num_objects":4973,"num_object_clones":0,"num_object_copies":9946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11988,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13221","reported_seq":14005,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.821954+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:33.821954+0000","last_peered":"2024-09-15T21:24:33.821954+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:33.821954+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10747,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13157","reported_seq":13582,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.684238+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:29.684238+0000","last_peered":"2024-09-15T21:24:29.684238+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:29.684238+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"41'3100","ondisk_log_start":"41'3100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11708,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2016}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13352","reported_seq":13764,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:29.709963+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:29.709963+0000","last_peered":"2024-09-15T21:24:29.709963+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:29.709963+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11751,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13663","reported_seq":14405,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822825+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:33.822825+0000","last_peered":"2024-09-15T21:24:33.822825+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:33.822825+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10970,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13446","reported_seq":17726,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.948771+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:33.948771+0000","last_peered":"2024-09-15T21:24:33.948771+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:33.948771+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13402","reported_seq":14061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.853331+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:33.853331+0000","last_peered":"2024-09-15T21:24:33.853331+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:33.853331+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13652","reported_seq":20746,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.302730+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:34.302730+0000","last_peered":"2024-09-15T21:24:34.302730+0000","last_clean":"2024-09-15T21:24:34.302730+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:34.302730+0000","last_undegraded":"2024-09-15T21:24:34.302730+0000","last_fullsized":"2024-09-15T21:24:34.302730+0000","mapping_epoch":47,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20341353,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330625665,"num_objects":80721,"num_object_clones":0,"num_object_copies":161442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80721,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187058,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14451,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":826867712,"data_stored":826803904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160744,"ondisk_log_size":160744,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953560,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537016,"kb_used_data":274856,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93834824,"statfs":{"total":96636764160,"available":96086859776,"internally_reserved":0,"allocated":281452544,"data_stored":281161687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673556,"kb_used_data":294832,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93698284,"statfs":{"total":96636764160,"available":95947042816,"internally_reserved":0,"allocated":301907968,"data_stored":301621183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,3,0,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557856,"kb_used_data":241540,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93813984,"statfs":{"total":96636764160,"available":96065519616,"internally_reserved":0,"allocated":247336960,"data_stored":247000876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300183552,"data_stored":300180073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245997568,"data_stored":245944790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280686592,"data_stored":280679041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:36.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:36.416 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13452", 'reported_seq': 14210, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.910197+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:33.910197+0000', 'last_peered': '2024-09-15T21:24:33.910197+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:33.910197+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'3400", 'ondisk_log_start': "44'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20725760, 'num_objects': 5060, 'num_object_clones': 0, 'num_object_copies': 10120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13452, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2029}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:36.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'13437", 'reported_seq': 14257, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.822733+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:33.822733+0000', 'last_peered': '2024-09-15T21:24:33.822733+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:33.822733+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'3400", 'ondisk_log_start': "44'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20250624, 'num_objects': 4944, 'num_object_clones': 0, 'num_object_copies': 9888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10862, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1915}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:36.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'13772", 'reported_seq': 14518, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.909425+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:33.909425+0000', 'last_peered': '2024-09-15T21:24:33.909425+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:33.909425+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21020672, 'num_objects': 5132, 'num_object_clones': 0, 'num_object_copies': 10264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11408, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:36.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'13221", 'reported_seq': 14005, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.821954+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:33.821954+0000', 'last_peered': '2024-09-15T21:24:33.821954+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:33.821954+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "41'3200", 'ondisk_log_start': "41'3200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19714048, 'num_objects': 4813, 'num_object_clones': 0, 'num_object_copies': 9626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10747, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1798}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:36.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'13157", 'reported_seq': 13582, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:29.684238+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:29.684238+0000', 'last_peered': '2024-09-15T21:24:29.684238+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:29.684238+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "41'3100", 'ondisk_log_start': "41'3100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21004288, 'num_objects': 5128, 'num_object_clones': 0, 'num_object_copies': 10256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11708, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2016}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:36.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:36.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13352", 'reported_seq': 13764, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:29.709963+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:29.709963+0000', 'last_peered': '2024-09-15T21:24:29.709963+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:29.709963+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "41'3300", 'ondisk_log_start': "41'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21282816, 'num_objects': 5196, 'num_object_clones': 0, 'num_object_copies': 10392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11751, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1990}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:36.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:36.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13663", 'reported_seq': 14405, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.822825+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:33.822825+0000', 'last_peered': '2024-09-15T21:24:33.822825+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:33.822825+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'3600", 'ondisk_log_start': "44'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20672512, 'num_objects': 5047, 'num_object_clones': 0, 'num_object_copies': 10094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10970, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:36.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'13446", 'reported_seq': 17726, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.948771+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:33.948771+0000', 'last_peered': '2024-09-15T21:24:33.948771+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:33.948771+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "44'3400", 'ondisk_log_start': "44'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20340736, 'num_objects': 4966, 'num_object_clones': 0, 'num_object_copies': 9932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1312, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11031, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2070, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3654}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:36.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'13402", 'reported_seq': 14061, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.853331+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:33.853331+0000', 'last_peered': '2024-09-15T21:24:33.853331+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:33.853331+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'3400", 'ondisk_log_start': "44'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20381696, 'num_objects': 4976, 'num_object_clones': 0, 'num_object_copies': 9952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11049, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:36.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:37.569 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:37.569 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26114/160200 objects degraded (16.301%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80100,"data_bytes":328073876,"bytes_used":1814409216,"bytes_avail":288095883264,"bytes_total":289910292480,"degraded_objects":26114,"degraded_total":160200,"degraded_ratio":0.16300873907615482,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":442},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-15T21:24:20.460939+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:37.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:37.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:38.456 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:38.458 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:38.520 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-15T21:24:36.464407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328073876,"num_objects":80100,"num_object_clones":0,"num_object_copies":160200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187683,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14452,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160570,"ondisk_log_size":160570,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1771884,"kb_used_data":806364,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281343636,"statfs":{"total":289910292480,"available":288095883264,"internally_reserved":0,"allocated":825716736,"data_stored":824803010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":988597258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,16,4,4,17,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21766144,"num_objects":-5314,"num_object_clones":0,"num_object_copies":-10628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5314,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002688"},"pg_stats":[{"pgid":"2.f","version":"50'13452","reported_seq":14210,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.910197+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:33.910197+0000","last_peered":"2024-09-15T21:24:33.910197+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:33.910197+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725760,"num_objects":5060,"num_object_clones":0,"num_object_copies":10120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13452,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13622","reported_seq":13728,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.690524+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:34.690524+0000","last_peered":"2024-09-15T21:24:34.690524+0000","last_clean":"2024-09-15T21:24:34.690524+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:34.690524+0000","last_undegraded":"2024-09-15T21:24:34.690524+0000","last_fullsized":"2024-09-15T21:24:34.690524+0000","mapping_epoch":36,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20553728,"num_objects":5018,"num_object_clones":0,"num_object_copies":10036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12348,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13648","reported_seq":20573,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.350678+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:34.350678+0000","last_peered":"2024-09-15T21:24:34.350678+0000","last_clean":"2024-09-15T21:24:34.350678+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:34.350678+0000","last_undegraded":"2024-09-15T21:24:34.350678+0000","last_fullsized":"2024-09-15T21:24:34.350678+0000","mapping_epoch":45,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385240,"num_objects":5222,"num_object_clones":0,"num_object_copies":10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12371,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13634","reported_seq":13713,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.725089+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:34.725089+0000","last_peered":"2024-09-15T21:24:34.725089+0000","last_clean":"2024-09-15T21:24:34.725089+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:34.725089+0000","last_undegraded":"2024-09-15T21:24:34.725089+0000","last_fullsized":"2024-09-15T21:24:34.725089+0000","mapping_epoch":36,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20217856,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13640","reported_seq":13768,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.881400+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:33.881400+0000","last_peered":"2024-09-15T21:24:33.881400+0000","last_clean":"2024-09-15T21:24:33.881400+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:33.881400+0000","last_undegraded":"2024-09-15T21:24:33.881400+0000","last_fullsized":"2024-09-15T21:24:33.881400+0000","mapping_epoch":35,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20819968,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13437","reported_seq":14257,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822733+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:33.822733+0000","last_peered":"2024-09-15T21:24:33.822733+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:33.822733+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10862,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13772","reported_seq":14518,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.909425+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:33.909425+0000","last_peered":"2024-09-15T21:24:33.909425+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:33.909425+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21020672,"num_objects":5132,"num_object_clones":0,"num_object_copies":10264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11408,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13469","reported_seq":13598,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.900349+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:33.900349+0000","last_peered":"2024-09-15T21:24:33.900349+0000","last_clean":"2024-09-15T21:24:33.900349+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:33.900349+0000","last_undegraded":"2024-09-15T21:24:33.900349+0000","last_fullsized":"2024-09-15T21:24:33.900349+0000","mapping_epoch":35,"log_start":"41'3400","ondisk_log_start":"41'3400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11254,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659110+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659110+0000","last_peered":"2024-09-15T21:23:34.659110+0000","last_clean":"2024-09-15T21:23:34.659110+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659110+0000","last_undegraded":"2024-09-15T21:23:34.659110+0000","last_fullsized":"2024-09-15T21:23:34.659110+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13402","reported_seq":20225,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.363454+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:34.363454+0000","last_peered":"2024-09-15T21:24:34.363454+0000","last_clean":"2024-09-15T21:24:34.363454+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:34.363454+0000","last_undegraded":"2024-09-15T21:24:34.363454+0000","last_fullsized":"2024-09-15T21:24:34.363454+0000","mapping_epoch":49,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20369408,"num_objects":4973,"num_object_clones":0,"num_object_copies":9946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11988,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13221","reported_seq":14005,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.821954+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:33.821954+0000","last_peered":"2024-09-15T21:24:33.821954+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:33.821954+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"41'3200","ondisk_log_start":"41'3200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10747,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13301","reported_seq":13728,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:34.668675+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:34.668675+0000","last_peered":"2024-09-15T21:24:34.668675+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:34.668675+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3300","ondisk_log_start":"44'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13506","reported_seq":13918,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:34.719349+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:34.719349+0000","last_peered":"2024-09-15T21:24:34.719349+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:34.719349+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20652032,"num_objects":5042,"num_object_clones":0,"num_object_copies":10084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11905,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1836}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13663","reported_seq":14405,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.822825+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:33.822825+0000","last_peered":"2024-09-15T21:24:33.822825+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:33.822825+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10970,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13446","reported_seq":17726,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.948771+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:33.948771+0000","last_peered":"2024-09-15T21:24:33.948771+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:33.948771+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20340736,"num_objects":4966,"num_object_clones":0,"num_object_copies":9932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11031,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13402","reported_seq":14061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.853331+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:33.853331+0000","last_peered":"2024-09-15T21:24:33.853331+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:33.853331+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13652","reported_seq":20746,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:34.302730+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:34.302730+0000","last_peered":"2024-09-15T21:24:34.302730+0000","last_clean":"2024-09-15T21:24:34.302730+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:34.302730+0000","last_undegraded":"2024-09-15T21:24:34.302730+0000","last_fullsized":"2024-09-15T21:24:34.302730+0000","mapping_epoch":47,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20341353,"num_objects":4967,"num_object_clones":0,"num_object_copies":9934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11312,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":328073857,"num_objects":80098,"num_object_clones":0,"num_object_copies":160196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187681,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14451,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":821936128,"data_stored":821872320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160567,"ondisk_log_size":160567,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953561,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540472,"kb_used_data":269992,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93831368,"statfs":{"total":96636764160,"available":96083320832,"internally_reserved":0,"allocated":276471808,"data_stored":276180951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149788,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673556,"kb_used_data":294832,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93698284,"statfs":{"total":96636764160,"available":95947042816,"internally_reserved":0,"allocated":301907968,"data_stored":301621183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,3,0,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557856,"kb_used_data":241540,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93813984,"statfs":{"total":96636764160,"available":96065519616,"internally_reserved":0,"allocated":247336960,"data_stored":247000876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":323874476},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":300183552,"data_stored":300180073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245997568,"data_stored":245944790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275755008,"data_stored":275747457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:41.524 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:42.132 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:42.167 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-15T21:24:40.467000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318419604,"num_objects":77743,"num_object_clones":0,"num_object_copies":155486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190040,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14606,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769684,"kb_used_data":787544,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281345836,"statfs":{"total":289910292480,"available":288098136064,"internally_reserved":0,"allocated":806445056,"data_stored":805531330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1005636618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,2,9,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29138944,"num_objects":-7114,"num_object_clones":0,"num_object_copies":-14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7114,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"50'13605","reported_seq":14363,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.982783+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:38.982783+0000","last_peered":"2024-09-15T21:24:38.982783+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:38.982783+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13605,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1876}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13768","reported_seq":13874,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.699924+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:39.699924+0000","last_peered":"2024-09-15T21:24:39.699924+0000","last_clean":"2024-09-15T21:24:39.699924+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:39.699924+0000","last_undegraded":"2024-09-15T21:24:39.699924+0000","last_fullsized":"2024-09-15T21:24:39.699924+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19955712,"num_objects":4872,"num_object_clones":0,"num_object_copies":9744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12494,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13809","reported_seq":20734,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.336681+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:39.336681+0000","last_peered":"2024-09-15T21:24:39.336681+0000","last_clean":"2024-09-15T21:24:39.336681+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:39.336681+0000","last_undegraded":"2024-09-15T21:24:39.336681+0000","last_fullsized":"2024-09-15T21:24:39.336681+0000","mapping_epoch":45,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725784,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12532,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13762","reported_seq":13841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.745582+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:39.745582+0000","last_peered":"2024-09-15T21:24:39.745582+0000","last_clean":"2024-09-15T21:24:39.745582+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:39.745582+0000","last_undegraded":"2024-09-15T21:24:39.745582+0000","last_fullsized":"2024-09-15T21:24:39.745582+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13786","reported_seq":13914,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.005707+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:39.005707+0000","last_peered":"2024-09-15T21:24:39.005707+0000","last_clean":"2024-09-15T21:24:39.005707+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:39.005707+0000","last_undegraded":"2024-09-15T21:24:39.005707+0000","last_fullsized":"2024-09-15T21:24:39.005707+0000","mapping_epoch":35,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20221952,"num_objects":4937,"num_object_clones":0,"num_object_copies":9874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11644,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13597","reported_seq":14420,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.889906+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:38.889906+0000","last_peered":"2024-09-15T21:24:38.889906+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:38.889906+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":4784,"num_object_clones":0,"num_object_copies":9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11022,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13928","reported_seq":14674,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.902208+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:38.902208+0000","last_peered":"2024-09-15T21:24:38.902208+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:38.902208+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11564,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13594","reported_seq":13723,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.966110+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:38.966110+0000","last_peered":"2024-09-15T21:24:38.966110+0000","last_clean":"2024-09-15T21:24:38.966110+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:38.966110+0000","last_undegraded":"2024-09-15T21:24:38.966110+0000","last_fullsized":"2024-09-15T21:24:38.966110+0000","mapping_epoch":35,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19701760,"num_objects":4810,"num_object_clones":0,"num_object_copies":9620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11379,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13537","reported_seq":20360,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.390482+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:39.390482+0000","last_peered":"2024-09-15T21:24:39.390482+0000","last_clean":"2024-09-15T21:24:39.390482+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:39.390482+0000","last_undegraded":"2024-09-15T21:24:39.390482+0000","last_fullsized":"2024-09-15T21:24:39.390482+0000","mapping_epoch":49,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13368","reported_seq":14150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.948446+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:38.948446+0000","last_peered":"2024-09-15T21:24:38.948446+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:38.948446+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19111936,"num_objects":4666,"num_object_clones":0,"num_object_copies":9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13458","reported_seq":13885,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.732898+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:39.732898+0000","last_peered":"2024-09-15T21:24:39.732898+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:39.732898+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19771392,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12009,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13686","reported_seq":14098,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.729026+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:39.729026+0000","last_peered":"2024-09-15T21:24:39.729026+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:39.729026+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1656}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13807","reported_seq":14549,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.958043+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:38.958043+0000","last_peered":"2024-09-15T21:24:38.958043+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:38.958043+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13593","reported_seq":18181,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.011348+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:39.011348+0000","last_peered":"2024-09-15T21:24:39.011348+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:39.011348+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19738624,"num_objects":4819,"num_object_clones":0,"num_object_copies":9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2224,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13541","reported_seq":14200,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.940303+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:38.940303+0000","last_peered":"2024-09-15T21:24:38.940303+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:38.940303+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19812352,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11188,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13785","reported_seq":20879,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.299879+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:39.299879+0000","last_peered":"2024-09-15T21:24:39.299879+0000","last_clean":"2024-09-15T21:24:39.299879+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:39.299879+0000","last_undegraded":"2024-09-15T21:24:39.299879+0000","last_fullsized":"2024-09-15T21:24:39.299879+0000","mapping_epoch":47,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19796585,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318419585,"num_objects":77741,"num_object_clones":0,"num_object_copies":155482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190038,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14605,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802635776,"data_stored":802571968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539984,"kb_used_data":265340,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93831856,"statfs":{"total":96636764160,"available":96083820544,"internally_reserved":0,"allocated":271708160,"data_stored":271417303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670668,"kb_used_data":287804,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93701172,"statfs":{"total":96636764160,"available":95950000128,"internally_reserved":0,"allocated":294711296,"data_stored":294424511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559032,"kb_used_data":234400,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93812808,"statfs":{"total":96636764160,"available":96064315392,"internally_reserved":0,"allocated":240025600,"data_stored":239689516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,2,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293044224,"data_stored":293040745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238649344,"data_stored":238596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270942208,"data_stored":270934657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:42.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:42.807 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:42.809 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:42.853 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-15T21:24:42.467494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318419604,"num_objects":77743,"num_object_clones":0,"num_object_copies":155486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190040,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14606,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769684,"kb_used_data":787544,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281345836,"statfs":{"total":289910292480,"available":288098136064,"internally_reserved":0,"allocated":806445056,"data_stored":805531330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1005636618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,2,9,11,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19636224,"num_objects":-4794,"num_object_clones":0,"num_object_copies":-9588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4794,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004538"},"pg_stats":[{"pgid":"2.f","version":"50'13605","reported_seq":14363,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.982783+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:38.982783+0000","last_peered":"2024-09-15T21:24:38.982783+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:38.982783+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13605,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1876}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13768","reported_seq":13874,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.699924+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:39.699924+0000","last_peered":"2024-09-15T21:24:39.699924+0000","last_clean":"2024-09-15T21:24:39.699924+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:39.699924+0000","last_undegraded":"2024-09-15T21:24:39.699924+0000","last_fullsized":"2024-09-15T21:24:39.699924+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19955712,"num_objects":4872,"num_object_clones":0,"num_object_copies":9744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12494,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13809","reported_seq":20734,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.336681+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:39.336681+0000","last_peered":"2024-09-15T21:24:39.336681+0000","last_clean":"2024-09-15T21:24:39.336681+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:39.336681+0000","last_undegraded":"2024-09-15T21:24:39.336681+0000","last_fullsized":"2024-09-15T21:24:39.336681+0000","mapping_epoch":45,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725784,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12532,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13762","reported_seq":13841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.745582+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:39.745582+0000","last_peered":"2024-09-15T21:24:39.745582+0000","last_clean":"2024-09-15T21:24:39.745582+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:39.745582+0000","last_undegraded":"2024-09-15T21:24:39.745582+0000","last_fullsized":"2024-09-15T21:24:39.745582+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13786","reported_seq":13914,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.005707+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:39.005707+0000","last_peered":"2024-09-15T21:24:39.005707+0000","last_clean":"2024-09-15T21:24:39.005707+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:39.005707+0000","last_undegraded":"2024-09-15T21:24:39.005707+0000","last_fullsized":"2024-09-15T21:24:39.005707+0000","mapping_epoch":35,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20221952,"num_objects":4937,"num_object_clones":0,"num_object_copies":9874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11644,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13597","reported_seq":14420,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.889906+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:38.889906+0000","last_peered":"2024-09-15T21:24:38.889906+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:38.889906+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":4784,"num_object_clones":0,"num_object_copies":9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11022,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'13928","reported_seq":14674,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.902208+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:38.902208+0000","last_peered":"2024-09-15T21:24:38.902208+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:38.902208+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20381696,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11564,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13594","reported_seq":13723,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.966110+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:38.966110+0000","last_peered":"2024-09-15T21:24:38.966110+0000","last_clean":"2024-09-15T21:24:38.966110+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:38.966110+0000","last_undegraded":"2024-09-15T21:24:38.966110+0000","last_fullsized":"2024-09-15T21:24:38.966110+0000","mapping_epoch":35,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19701760,"num_objects":4810,"num_object_clones":0,"num_object_copies":9620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11379,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13537","reported_seq":20360,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.390482+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:39.390482+0000","last_peered":"2024-09-15T21:24:39.390482+0000","last_clean":"2024-09-15T21:24:39.390482+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:39.390482+0000","last_undegraded":"2024-09-15T21:24:39.390482+0000","last_fullsized":"2024-09-15T21:24:39.390482+0000","mapping_epoch":49,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13368","reported_seq":14150,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.948446+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:38.948446+0000","last_peered":"2024-09-15T21:24:38.948446+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:38.948446+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"41'3300","ondisk_log_start":"41'3300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19111936,"num_objects":4666,"num_object_clones":0,"num_object_copies":9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13458","reported_seq":13885,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.732898+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:39.732898+0000","last_peered":"2024-09-15T21:24:39.732898+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:39.732898+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19771392,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12009,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13686","reported_seq":14098,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.729026+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:39.729026+0000","last_peered":"2024-09-15T21:24:39.729026+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:39.729026+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1656}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13807","reported_seq":14549,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.958043+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:38.958043+0000","last_peered":"2024-09-15T21:24:38.958043+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:38.958043+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13593","reported_seq":18181,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.011348+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:39.011348+0000","last_peered":"2024-09-15T21:24:39.011348+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:39.011348+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19738624,"num_objects":4819,"num_object_clones":0,"num_object_copies":9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2224,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3661}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13541","reported_seq":14200,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.940303+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:38.940303+0000","last_peered":"2024-09-15T21:24:38.940303+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:38.940303+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19812352,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11188,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13785","reported_seq":20879,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.299879+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:39.299879+0000","last_peered":"2024-09-15T21:24:39.299879+0000","last_clean":"2024-09-15T21:24:39.299879+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:39.299879+0000","last_undegraded":"2024-09-15T21:24:39.299879+0000","last_fullsized":"2024-09-15T21:24:39.299879+0000","mapping_epoch":47,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19796585,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318419585,"num_objects":77741,"num_object_clones":0,"num_object_copies":155482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190038,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14605,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802635776,"data_stored":802571968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539984,"kb_used_data":265340,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93831856,"statfs":{"total":96636764160,"available":96083820544,"internally_reserved":0,"allocated":271708160,"data_stored":271417303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670668,"kb_used_data":287804,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93701172,"statfs":{"total":96636764160,"available":95950000128,"internally_reserved":0,"allocated":294711296,"data_stored":294424511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559032,"kb_used_data":234400,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93812808,"statfs":{"total":96636764160,"available":96064315392,"internally_reserved":0,"allocated":240025600,"data_stored":239689516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,2,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293044224,"data_stored":293040745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238649344,"data_stored":238596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270942208,"data_stored":270934657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:42.854 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13605", 'reported_seq': 14363, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.982783+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:38.982783+0000', 'last_peered': '2024-09-15T21:24:38.982783+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:38.982783+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'3600", 'ondisk_log_start': "44'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20099072, 'num_objects': 4907, 'num_object_clones': 0, 'num_object_copies': 9814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13605, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1876}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'13597", 'reported_seq': 14420, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.889906+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:38.889906+0000', 'last_peered': '2024-09-15T21:24:38.889906+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:38.889906+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'3500", 'ondisk_log_start': "44'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19595264, 'num_objects': 4784, 'num_object_clones': 0, 'num_object_copies': 9568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11022, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'13928", 'reported_seq': 14674, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.902208+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:38.902208+0000', 'last_peered': '2024-09-15T21:24:38.902208+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:38.902208+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20381696, 'num_objects': 4976, 'num_object_clones': 0, 'num_object_copies': 9952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11564, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'13368", 'reported_seq': 14150, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.948446+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:38.948446+0000', 'last_peered': '2024-09-15T21:24:38.948446+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:38.948446+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "41'3300", 'ondisk_log_start': "41'3300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19111936, 'num_objects': 4666, 'num_object_clones': 0, 'num_object_copies': 9332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10894, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1651}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'13458", 'reported_seq': 13885, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:39.732898+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:39.732898+0000', 'last_peered': '2024-09-15T21:24:39.732898+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:39.732898+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'3400", 'ondisk_log_start': "44'3400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19771392, 'num_objects': 4827, 'num_object_clones': 0, 'num_object_copies': 9654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12009, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1715}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:42.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13686", 'reported_seq': 14098, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:39.729026+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:39.729026+0000', 'last_peered': '2024-09-15T21:24:39.729026+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:39.729026+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'3600", 'ondisk_log_start': "44'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19914752, 'num_objects': 4862, 'num_object_clones': 0, 'num_object_copies': 9724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12085, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1656}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13807", 'reported_seq': 14549, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.958043+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:38.958043+0000', 'last_peered': '2024-09-15T21:24:38.958043+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:38.958043+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'3800", 'ondisk_log_start': "44'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20082688, 'num_objects': 4903, 'num_object_clones': 0, 'num_object_copies': 9806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11114, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1773}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'13593", 'reported_seq': 18181, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:39.011348+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:39.011348+0000', 'last_peered': '2024-09-15T21:24:39.011348+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:39.011348+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "44'3500", 'ondisk_log_start': "44'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19738624, 'num_objects': 4819, 'num_object_clones': 0, 'num_object_copies': 9638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1158, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2224, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3661}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:42.856 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'13541", 'reported_seq': 14200, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.940303+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:38.940303+0000', 'last_peered': '2024-09-15T21:24:38.940303+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:38.940303+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'3500", 'ondisk_log_start': "44'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19812352, 'num_objects': 4837, 'num_object_clones': 0, 'num_object_copies': 9674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11188, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:42.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:44.016 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:44.017 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25960/155486 objects degraded (16.696%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77743,"data_bytes":318419604,"bytes_used":1812156416,"bytes_avail":288098136064,"bytes_total":289910292480,"degraded_objects":25960,"degraded_total":155486,"degraded_ratio":0.16696036942232742,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":399},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:44.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:44.134 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:44.898 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:44.899 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:44.922 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-15T21:24:44.472417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":313082516,"num_objects":76440,"num_object_clones":0,"num_object_copies":152880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76440,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191343,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1754636,"kb_used_data":780816,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":281360884,"statfs":{"total":289910292480,"available":288113545216,"internally_reserved":0,"allocated":799555584,"data_stored":798641858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":997116938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,6,6,10,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24973312,"num_objects":-6097,"num_object_clones":0,"num_object_copies":-12194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6097,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008946"},"pg_stats":[{"pgid":"2.f","version":"50'13749","reported_seq":14507,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.009344+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:44.009344+0000","last_peered":"2024-09-15T21:24:44.009344+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:44.009344+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13749,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13768","reported_seq":13874,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.699924+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:39.699924+0000","last_peered":"2024-09-15T21:24:39.699924+0000","last_clean":"2024-09-15T21:24:39.699924+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:39.699924+0000","last_undegraded":"2024-09-15T21:24:39.699924+0000","last_fullsized":"2024-09-15T21:24:39.699924+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19955712,"num_objects":4872,"num_object_clones":0,"num_object_copies":9744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12494,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13809","reported_seq":20734,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.336681+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:39.336681+0000","last_peered":"2024-09-15T21:24:39.336681+0000","last_clean":"2024-09-15T21:24:39.336681+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:39.336681+0000","last_undegraded":"2024-09-15T21:24:39.336681+0000","last_fullsized":"2024-09-15T21:24:39.336681+0000","mapping_epoch":45,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20725784,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12532,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13762","reported_seq":13841,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.745582+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:39.745582+0000","last_peered":"2024-09-15T21:24:39.745582+0000","last_clean":"2024-09-15T21:24:39.745582+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:39.745582+0000","last_undegraded":"2024-09-15T21:24:39.745582+0000","last_fullsized":"2024-09-15T21:24:39.745582+0000","mapping_epoch":36,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13955","reported_seq":14083,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.027736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:44.027736+0000","last_peered":"2024-09-15T21:24:44.027736+0000","last_clean":"2024-09-15T21:24:44.027736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:44.027736+0000","last_undegraded":"2024-09-15T21:24:44.027736+0000","last_fullsized":"2024-09-15T21:24:44.027736+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11813,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13743","reported_seq":14566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.067518+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:44.067518+0000","last_peered":"2024-09-15T21:24:44.067518+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:44.067518+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14057","reported_seq":14803,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.071665+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:44.071665+0000","last_peered":"2024-09-15T21:24:44.071665+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:44.071665+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13735","reported_seq":13864,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.067605+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:44.067605+0000","last_peered":"2024-09-15T21:24:44.067605+0000","last_clean":"2024-09-15T21:24:44.067605+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:44.067605+0000","last_undegraded":"2024-09-15T21:24:44.067605+0000","last_fullsized":"2024-09-15T21:24:44.067605+0000","mapping_epoch":35,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19124224,"num_objects":4669,"num_object_clones":0,"num_object_copies":9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11520,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13537","reported_seq":20360,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.390482+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:39.390482+0000","last_peered":"2024-09-15T21:24:39.390482+0000","last_clean":"2024-09-15T21:24:39.390482+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:39.390482+0000","last_undegraded":"2024-09-15T21:24:39.390482+0000","last_fullsized":"2024-09-15T21:24:39.390482+0000","mapping_epoch":49,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13510","reported_seq":14293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.065881+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:44.065881+0000","last_peered":"2024-09-15T21:24:44.065881+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:44.065881+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18530304,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11036,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1509}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13458","reported_seq":13885,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.732898+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:39.732898+0000","last_peered":"2024-09-15T21:24:39.732898+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:39.732898+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3400","ondisk_log_start":"44'3400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19771392,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12009,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13686","reported_seq":14098,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:39.729026+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:39.729026+0000","last_peered":"2024-09-15T21:24:39.729026+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:39.729026+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19914752,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1656}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13934","reported_seq":14677,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.072050+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:44.072050+0000","last_peered":"2024-09-15T21:24:44.072050+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:44.072050+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19562496,"num_objects":4776,"num_object_clones":0,"num_object_copies":9552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11241,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13740","reported_seq":18708,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.073590+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:44.073590+0000","last_peered":"2024-09-15T21:24:44.073590+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:44.073590+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":4672,"num_object_clones":0,"num_object_copies":9344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11325,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13699","reported_seq":14358,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.023975+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:44.023975+0000","last_peered":"2024-09-15T21:24:44.023975+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:44.023975+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19165184,"num_objects":4679,"num_object_clones":0,"num_object_copies":9358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13785","reported_seq":20879,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:39.299879+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:39.299879+0000","last_peered":"2024-09-15T21:24:39.299879+0000","last_clean":"2024-09-15T21:24:39.299879+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:39.299879+0000","last_undegraded":"2024-09-15T21:24:39.299879+0000","last_fullsized":"2024-09-15T21:24:39.299879+0000","mapping_epoch":47,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19796585,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11445,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":313082497,"num_objects":76438,"num_object_clones":0,"num_object_copies":152876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191341,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14795,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":795500544,"data_stored":795436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953562,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539984,"kb_used_data":265340,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93831856,"statfs":{"total":96636764160,"available":96083820544,"internally_reserved":0,"allocated":271708160,"data_stored":271417303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655620,"kb_used_data":281076,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93716220,"statfs":{"total":96636764160,"available":95965409280,"internally_reserved":0,"allocated":287821824,"data_stored":287535039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559032,"kb_used_data":234400,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93812808,"statfs":{"total":96636764160,"available":96064315392,"internally_reserved":0,"allocated":240025600,"data_stored":239689516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":332394156},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,2,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285905513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238649344,"data_stored":238596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270942208,"data_stored":270934657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:47.924 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:48.523 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:48.523 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:48.553 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-15T21:24:48.474969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308949652,"num_objects":75431,"num_object_clones":0,"num_object_copies":150862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192352,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755564,"kb_used_data":769276,"kb_used_omap":12,"kb_used_meta":986227,"kb_avail":281359956,"statfs":{"total":289910292480,"available":288112594944,"internally_reserved":0,"allocated":787738624,"data_stored":786824898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1009896458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,10,7,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010561"},"pg_stats":[{"pgid":"2.f","version":"50'13749","reported_seq":14507,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.009344+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:44.009344+0000","last_peered":"2024-09-15T21:24:44.009344+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:44.009344+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13749,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13923","reported_seq":14029,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.647048+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:44.647048+0000","last_peered":"2024-09-15T21:24:44.647048+0000","last_clean":"2024-09-15T21:24:44.647048+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:44.647048+0000","last_undegraded":"2024-09-15T21:24:44.647048+0000","last_fullsized":"2024-09-15T21:24:44.647048+0000","mapping_epoch":36,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19320832,"num_objects":4717,"num_object_clones":0,"num_object_copies":9434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13974","reported_seq":20899,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.367162+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:44.367162+0000","last_peered":"2024-09-15T21:24:44.367162+0000","last_clean":"2024-09-15T21:24:44.367162+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:44.367162+0000","last_undegraded":"2024-09-15T21:24:44.367162+0000","last_fullsized":"2024-09-15T21:24:44.367162+0000","mapping_epoch":45,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049944,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12697,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13893","reported_seq":13972,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.753990+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:44.753990+0000","last_peered":"2024-09-15T21:24:44.753990+0000","last_clean":"2024-09-15T21:24:44.753990+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:44.753990+0000","last_undegraded":"2024-09-15T21:24:44.753990+0000","last_fullsized":"2024-09-15T21:24:44.753990+0000","mapping_epoch":36,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13893,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13955","reported_seq":14083,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.027736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:44.027736+0000","last_peered":"2024-09-15T21:24:44.027736+0000","last_clean":"2024-09-15T21:24:44.027736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:44.027736+0000","last_undegraded":"2024-09-15T21:24:44.027736+0000","last_fullsized":"2024-09-15T21:24:44.027736+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11813,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13743","reported_seq":14566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.067518+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:44.067518+0000","last_peered":"2024-09-15T21:24:44.067518+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:44.067518+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14057","reported_seq":14803,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.071665+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:44.071665+0000","last_peered":"2024-09-15T21:24:44.071665+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:44.071665+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13735","reported_seq":13864,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.067605+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:44.067605+0000","last_peered":"2024-09-15T21:24:44.067605+0000","last_clean":"2024-09-15T21:24:44.067605+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:44.067605+0000","last_undegraded":"2024-09-15T21:24:44.067605+0000","last_fullsized":"2024-09-15T21:24:44.067605+0000","mapping_epoch":35,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19124224,"num_objects":4669,"num_object_clones":0,"num_object_copies":9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11520,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13673","reported_seq":20496,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.440131+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:44.440131+0000","last_peered":"2024-09-15T21:24:44.440131+0000","last_clean":"2024-09-15T21:24:44.440131+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:44.440131+0000","last_undegraded":"2024-09-15T21:24:44.440131+0000","last_fullsized":"2024-09-15T21:24:44.440131+0000","mapping_epoch":49,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12259,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13510","reported_seq":14293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.065881+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:44.065881+0000","last_peered":"2024-09-15T21:24:44.065881+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:44.065881+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18530304,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11036,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1509}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13599","reported_seq":14026,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.713255+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:44.713255+0000","last_peered":"2024-09-15T21:24:44.713255+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:44.713255+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193856,"num_objects":4686,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12150,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1574}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13824","reported_seq":14236,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.679615+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:44.679615+0000","last_peered":"2024-09-15T21:24:44.679615+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:44.679615+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349504,"num_objects":4724,"num_object_clones":0,"num_object_copies":9448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12223,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1518}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13934","reported_seq":14677,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.072050+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:44.072050+0000","last_peered":"2024-09-15T21:24:44.072050+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:44.072050+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19562496,"num_objects":4776,"num_object_clones":0,"num_object_copies":9552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11241,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13740","reported_seq":18708,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.073590+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:44.073590+0000","last_peered":"2024-09-15T21:24:44.073590+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:44.073590+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":4672,"num_object_clones":0,"num_object_copies":9344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11325,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13699","reported_seq":14358,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.023975+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:44.023975+0000","last_peered":"2024-09-15T21:24:44.023975+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:44.023975+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19165184,"num_objects":4679,"num_object_clones":0,"num_object_copies":9358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13928","reported_seq":21022,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.324590+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:44.324590+0000","last_peered":"2024-09-15T21:24:44.324590+0000","last_clean":"2024-09-15T21:24:44.324590+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:44.324590+0000","last_undegraded":"2024-09-15T21:24:44.324590+0000","last_fullsized":"2024-09-15T21:24:44.324590+0000","mapping_epoch":47,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19210857,"num_objects":4691,"num_object_clones":0,"num_object_copies":9382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11588,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308949633,"num_objects":75429,"num_object_clones":0,"num_object_copies":150858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192350,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14795,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783663104,"data_stored":783599296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953563,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539412,"kb_used_data":260612,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93832428,"statfs":{"total":96636764160,"available":96084406272,"internally_reserved":0,"allocated":266866688,"data_stored":266575831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":285470381},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655620,"kb_used_data":281076,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93716220,"statfs":{"total":96636764160,"available":95965409280,"internally_reserved":0,"allocated":287821824,"data_stored":287535039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560532,"kb_used_data":227588,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93811308,"statfs":{"total":96636764160,"available":96062779392,"internally_reserved":0,"allocated":233050112,"data_stored":232714028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":340913836},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285905513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231612416,"data_stored":231559638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266141696,"data_stored":266134145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:48.555 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:49.355 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:49.356 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:49.423 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-15T21:24:48.474969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308949652,"num_objects":75431,"num_object_clones":0,"num_object_copies":150862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192352,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14796,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755564,"kb_used_data":769276,"kb_used_omap":12,"kb_used_meta":986227,"kb_avail":281359956,"statfs":{"total":289910292480,"available":288112594944,"internally_reserved":0,"allocated":787738624,"data_stored":786824898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1009896458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,10,7,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010561"},"pg_stats":[{"pgid":"2.f","version":"50'13749","reported_seq":14507,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.009344+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:44.009344+0000","last_peered":"2024-09-15T21:24:44.009344+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:44.009344+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19509248,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13749,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1732}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'13923","reported_seq":14029,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.647048+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:44.647048+0000","last_peered":"2024-09-15T21:24:44.647048+0000","last_clean":"2024-09-15T21:24:44.647048+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:44.647048+0000","last_undegraded":"2024-09-15T21:24:44.647048+0000","last_fullsized":"2024-09-15T21:24:44.647048+0000","mapping_epoch":36,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19320832,"num_objects":4717,"num_object_clones":0,"num_object_copies":9434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'13974","reported_seq":20899,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.367162+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:44.367162+0000","last_peered":"2024-09-15T21:24:44.367162+0000","last_clean":"2024-09-15T21:24:44.367162+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:44.367162+0000","last_undegraded":"2024-09-15T21:24:44.367162+0000","last_fullsized":"2024-09-15T21:24:44.367162+0000","mapping_epoch":45,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049944,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12697,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'13893","reported_seq":13972,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.753990+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:44.753990+0000","last_peered":"2024-09-15T21:24:44.753990+0000","last_clean":"2024-09-15T21:24:44.753990+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:44.753990+0000","last_undegraded":"2024-09-15T21:24:44.753990+0000","last_fullsized":"2024-09-15T21:24:44.753990+0000","mapping_epoch":36,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13893,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'13955","reported_seq":14083,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.027736+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:44.027736+0000","last_peered":"2024-09-15T21:24:44.027736+0000","last_clean":"2024-09-15T21:24:44.027736+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:44.027736+0000","last_undegraded":"2024-09-15T21:24:44.027736+0000","last_fullsized":"2024-09-15T21:24:44.027736+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11813,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13743","reported_seq":14566,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.067518+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:44.067518+0000","last_peered":"2024-09-15T21:24:44.067518+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:44.067518+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1609}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14057","reported_seq":14803,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.071665+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:44.071665+0000","last_peered":"2024-09-15T21:24:44.071665+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:44.071665+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13735","reported_seq":13864,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.067605+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:44.067605+0000","last_peered":"2024-09-15T21:24:44.067605+0000","last_clean":"2024-09-15T21:24:44.067605+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:44.067605+0000","last_undegraded":"2024-09-15T21:24:44.067605+0000","last_fullsized":"2024-09-15T21:24:44.067605+0000","mapping_epoch":35,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19124224,"num_objects":4669,"num_object_clones":0,"num_object_copies":9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11520,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13673","reported_seq":20496,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.440131+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:44.440131+0000","last_peered":"2024-09-15T21:24:44.440131+0000","last_clean":"2024-09-15T21:24:44.440131+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:44.440131+0000","last_undegraded":"2024-09-15T21:24:44.440131+0000","last_fullsized":"2024-09-15T21:24:44.440131+0000","mapping_epoch":49,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12259,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13510","reported_seq":14293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.065881+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:44.065881+0000","last_peered":"2024-09-15T21:24:44.065881+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:44.065881+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18530304,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11036,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1509}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13599","reported_seq":14026,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.713255+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:44.713255+0000","last_peered":"2024-09-15T21:24:44.713255+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:44.713255+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3500","ondisk_log_start":"44'3500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19193856,"num_objects":4686,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12150,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1574}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13824","reported_seq":14236,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.679615+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:44.679615+0000","last_peered":"2024-09-15T21:24:44.679615+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:44.679615+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349504,"num_objects":4724,"num_object_clones":0,"num_object_copies":9448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12223,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1518}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'13934","reported_seq":14677,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.072050+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:44.072050+0000","last_peered":"2024-09-15T21:24:44.072050+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:44.072050+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19562496,"num_objects":4776,"num_object_clones":0,"num_object_copies":9552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11241,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13740","reported_seq":18708,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.073590+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:44.073590+0000","last_peered":"2024-09-15T21:24:44.073590+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:44.073590+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":4672,"num_object_clones":0,"num_object_copies":9344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11325,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2414,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13699","reported_seq":14358,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:44.023975+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:44.023975+0000","last_peered":"2024-09-15T21:24:44.023975+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:44.023975+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19165184,"num_objects":4679,"num_object_clones":0,"num_object_copies":9358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11346,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'13928","reported_seq":21022,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:44.324590+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:44.324590+0000","last_peered":"2024-09-15T21:24:44.324590+0000","last_clean":"2024-09-15T21:24:44.324590+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:44.324590+0000","last_undegraded":"2024-09-15T21:24:44.324590+0000","last_fullsized":"2024-09-15T21:24:44.324590+0000","mapping_epoch":47,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19210857,"num_objects":4691,"num_object_clones":0,"num_object_copies":9382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11588,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308949633,"num_objects":75429,"num_object_clones":0,"num_object_copies":150858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25770,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192350,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14795,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783663104,"data_stored":783599296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953563,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539412,"kb_used_data":260612,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93832428,"statfs":{"total":96636764160,"available":96084406272,"internally_reserved":0,"allocated":266866688,"data_stored":266575831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":285470381},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655620,"kb_used_data":281076,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93716220,"statfs":{"total":96636764160,"available":95965409280,"internally_reserved":0,"allocated":287821824,"data_stored":287535039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560532,"kb_used_data":227588,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93811308,"statfs":{"total":96636764160,"available":96062779392,"internally_reserved":0,"allocated":233050112,"data_stored":232714028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":340913836},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285908992,"data_stored":285905513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231612416,"data_stored":231559638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266141696,"data_stored":266134145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13749", 'reported_seq': 14507, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.009344+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:44.009344+0000', 'last_peered': '2024-09-15T21:24:44.009344+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:44.009344+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19509248, 'num_objects': 4763, 'num_object_clones': 0, 'num_object_copies': 9526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13749, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1732}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'13743", 'reported_seq': 14566, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.067518+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:44.067518+0000', 'last_peered': '2024-09-15T21:24:44.067518+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:44.067518+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18997248, 'num_objects': 4638, 'num_object_clones': 0, 'num_object_copies': 9276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11168, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1609}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'14057", 'reported_seq': 14803, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.071665+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:44.071665+0000', 'last_peered': '2024-09-15T21:24:44.071665+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:44.071665+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19853312, 'num_objects': 4847, 'num_object_clones': 0, 'num_object_copies': 9694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11693, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'13510", 'reported_seq': 14293, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.065881+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:44.065881+0000', 'last_peered': '2024-09-15T21:24:44.065881+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:44.065881+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'3500", 'ondisk_log_start': "44'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18530304, 'num_objects': 4524, 'num_object_clones': 0, 'num_object_copies': 9048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11036, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1509}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'13599", 'reported_seq': 14026, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.713255+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:44.713255+0000', 'last_peered': '2024-09-15T21:24:44.713255+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:44.713255+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'3500", 'ondisk_log_start': "44'3500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19193856, 'num_objects': 4686, 'num_object_clones': 0, 'num_object_copies': 9372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12150, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1574}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13824", 'reported_seq': 14236, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.679615+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:44.679615+0000', 'last_peered': '2024-09-15T21:24:44.679615+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:44.679615+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'3800", 'ondisk_log_start': "44'3800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19349504, 'num_objects': 4724, 'num_object_clones': 0, 'num_object_copies': 9448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12223, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1518}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'13934", 'reported_seq': 14677, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.072050+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:44.072050+0000', 'last_peered': '2024-09-15T21:24:44.072050+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:44.072050+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19562496, 'num_objects': 4776, 'num_object_clones': 0, 'num_object_copies': 9552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11241, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1646}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'13740", 'reported_seq': 18708, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.073590+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:44.073590+0000', 'last_peered': '2024-09-15T21:24:44.073590+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:44.073590+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19136512, 'num_objects': 4672, 'num_object_clones': 0, 'num_object_copies': 9344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 968, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11325, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2414, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:49.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'13699", 'reported_seq': 14358, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:44.023975+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:44.023975+0000', 'last_peered': '2024-09-15T21:24:44.023975+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:44.023975+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'3600", 'ondisk_log_start': "44'3600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19165184, 'num_objects': 4679, 'num_object_clones': 0, 'num_object_copies': 9358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11346, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:49.427 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:50.448 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:50.449 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25770/150862 objects degraded (17.082%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75431,"data_bytes":308949652,"bytes_used":1797697536,"bytes_avail":288112594944,"bytes_total":289910292480,"degraded_objects":25770,"degraded_total":150862,"degraded_ratio":0.17081836380268059,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":344,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:50.525 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:50.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:51.417 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:51.419 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:51.478 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-15T21:24:50.475382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299881108,"num_objects":73217,"num_object_clones":0,"num_object_copies":146434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194566,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15045,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1716604,"kb_used_data":751112,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281398916,"statfs":{"total":289910292480,"available":288152489984,"internally_reserved":0,"allocated":769138688,"data_stored":768224962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":988597258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,9,6,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28192768,"num_objects":-6883,"num_object_clones":0,"num_object_copies":-13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-593,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6883,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":593,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008819"},"pg_stats":[{"pgid":"2.f","version":"50'13875","reported_seq":14632,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.139899+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:49.139899+0000","last_peered":"2024-09-15T21:24:49.139899+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:49.139899+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1606}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14061","reported_seq":14167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.524861+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:49.524861+0000","last_peered":"2024-09-15T21:24:49.524861+0000","last_clean":"2024-09-15T21:24:49.524861+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:49.524861+0000","last_undegraded":"2024-09-15T21:24:49.524861+0000","last_fullsized":"2024-09-15T21:24:49.524861+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12787,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14116","reported_seq":21040,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.440992+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:49.440992+0000","last_peered":"2024-09-15T21:24:49.440992+0000","last_clean":"2024-09-15T21:24:49.440992+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:49.440992+0000","last_undegraded":"2024-09-15T21:24:49.440992+0000","last_fullsized":"2024-09-15T21:24:49.440992+0000","mapping_epoch":45,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468312,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14028","reported_seq":14107,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.732692+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:49.732692+0000","last_peered":"2024-09-15T21:24:49.732692+0000","last_clean":"2024-09-15T21:24:49.732692+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:49.732692+0000","last_undegraded":"2024-09-15T21:24:49.732692+0000","last_fullsized":"2024-09-15T21:24:49.732692+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14098","reported_seq":14226,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.039046+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:49.039046+0000","last_peered":"2024-09-15T21:24:49.039046+0000","last_clean":"2024-09-15T21:24:49.039046+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:49.039046+0000","last_undegraded":"2024-09-15T21:24:49.039046+0000","last_fullsized":"2024-09-15T21:24:49.039046+0000","mapping_epoch":35,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18944000,"num_objects":4625,"num_object_clones":0,"num_object_copies":9250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11956,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'13899","reported_seq":14722,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.115943+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:49.115943+0000","last_peered":"2024-09-15T21:24:49.115943+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:49.115943+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18358272,"num_objects":4482,"num_object_clones":0,"num_object_copies":8964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11324,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14210","reported_seq":14955,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.079079+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:49.079079+0000","last_peered":"2024-09-15T21:24:49.079079+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:49.079079+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19226624,"num_objects":4694,"num_object_clones":0,"num_object_copies":9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11846,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13862","reported_seq":13991,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.100198+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:49.100198+0000","last_peered":"2024-09-15T21:24:49.100198+0000","last_clean":"2024-09-15T21:24:49.100198+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:49.100198+0000","last_undegraded":"2024-09-15T21:24:49.100198+0000","last_fullsized":"2024-09-15T21:24:49.100198+0000","mapping_epoch":35,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11647,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13809","reported_seq":20631,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.520416+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:49.520416+0000","last_peered":"2024-09-15T21:24:49.520416+0000","last_clean":"2024-09-15T21:24:49.520416+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:49.520416+0000","last_undegraded":"2024-09-15T21:24:49.520416+0000","last_fullsized":"2024-09-15T21:24:49.520416+0000","mapping_epoch":49,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12395,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13642","reported_seq":14426,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.132740+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:49.132740+0000","last_peered":"2024-09-15T21:24:49.132740+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:49.132740+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3600","ondisk_log_start":"44'3600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17989632,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11168,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13732","reported_seq":14157,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.685649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:49.685649+0000","last_peered":"2024-09-15T21:24:49.685649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:49.685649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12283,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1441}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13960","reported_seq":14370,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.758005+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:49.758005+0000","last_peered":"2024-09-15T21:24:49.758005+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:49.758005+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18792448,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14063","reported_seq":14806,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.122521+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:49.122521+0000","last_peered":"2024-09-15T21:24:49.122521+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:49.122521+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19034112,"num_objects":4647,"num_object_clones":0,"num_object_copies":9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11370,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'13864","reported_seq":19330,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.139032+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:49.139032+0000","last_peered":"2024-09-15T21:24:49.139032+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:49.139032+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18628608,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11449,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2663,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'13858","reported_seq":14517,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.974275+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:48.974275+0000","last_peered":"2024-09-15T21:24:48.974275+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:48.974275+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14073","reported_seq":21167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.480159+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:49.480159+0000","last_peered":"2024-09-15T21:24:49.480159+0000","last_clean":"2024-09-15T21:24:49.480159+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:49.480159+0000","last_undegraded":"2024-09-15T21:24:49.480159+0000","last_fullsized":"2024-09-15T21:24:49.480159+0000","mapping_epoch":47,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18616937,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11733,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299881089,"num_objects":73215,"num_object_clones":0,"num_object_copies":146430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":194564,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15044,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":765517824,"data_stored":765454016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522500,"kb_used_data":256180,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93849340,"statfs":{"total":96636764160,"available":96101724160,"internally_reserved":0,"allocated":262328320,"data_stored":262037463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149791,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652676,"kb_used_data":273968,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93719164,"statfs":{"total":96636764160,"available":95968423936,"internally_reserved":0,"allocated":280543232,"data_stored":280256447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,3,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541428,"kb_used_data":220964,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93830412,"statfs":{"total":96636764160,"available":96082341888,"internally_reserved":0,"allocated":226267136,"data_stored":225931052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279056384,"data_stored":279052905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224780288,"data_stored":224727510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261681152,"data_stored":261673601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:54.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:55.053 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:55.102 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-15T21:24:54.476391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":294793876,"num_objects":71975,"num_object_clones":0,"num_object_copies":143950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195808,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15293,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160795,"ondisk_log_size":160795,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714128,"kb_used_data":744480,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281401392,"statfs":{"total":289910292480,"available":288155025408,"internally_reserved":0,"allocated":762347520,"data_stored":761433794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":992857098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,7,5,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23625728,"num_objects":-5768,"num_object_clones":0,"num_object_copies":-11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5768,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008896"},"pg_stats":[{"pgid":"2.f","version":"50'13997","reported_seq":14755,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.175903+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:54.175903+0000","last_peered":"2024-09-15T21:24:54.175903+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:54.175903+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493440,"num_objects":4515,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1484}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14061","reported_seq":14167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.524861+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:49.524861+0000","last_peered":"2024-09-15T21:24:49.524861+0000","last_clean":"2024-09-15T21:24:49.524861+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:49.524861+0000","last_undegraded":"2024-09-15T21:24:49.524861+0000","last_fullsized":"2024-09-15T21:24:49.524861+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12787,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14116","reported_seq":21040,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.440992+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:49.440992+0000","last_peered":"2024-09-15T21:24:49.440992+0000","last_clean":"2024-09-15T21:24:49.440992+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:49.440992+0000","last_undegraded":"2024-09-15T21:24:49.440992+0000","last_fullsized":"2024-09-15T21:24:49.440992+0000","mapping_epoch":45,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468312,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14028","reported_seq":14107,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.732692+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:49.732692+0000","last_peered":"2024-09-15T21:24:49.732692+0000","last_clean":"2024-09-15T21:24:49.732692+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:49.732692+0000","last_undegraded":"2024-09-15T21:24:49.732692+0000","last_fullsized":"2024-09-15T21:24:49.732692+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14254","reported_seq":14382,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.198509+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:54.198509+0000","last_peered":"2024-09-15T21:24:54.198509+0000","last_clean":"2024-09-15T21:24:54.198509+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:54.198509+0000","last_undegraded":"2024-09-15T21:24:54.198509+0000","last_fullsized":"2024-09-15T21:24:54.198509+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14019","reported_seq":14842,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.780696+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:53.780696+0000","last_peered":"2024-09-15T21:24:53.780696+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:53.780696+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14360","reported_seq":15105,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.143949+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:54.143949+0000","last_peered":"2024-09-15T21:24:54.143949+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:54.143949+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11996,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13976","reported_seq":14105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.121561+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:54.121561+0000","last_peered":"2024-09-15T21:24:54.121561+0000","last_clean":"2024-09-15T21:24:54.121561+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:54.121561+0000","last_undegraded":"2024-09-15T21:24:54.121561+0000","last_fullsized":"2024-09-15T21:24:54.121561+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13809","reported_seq":20631,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.520416+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:49.520416+0000","last_peered":"2024-09-15T21:24:49.520416+0000","last_clean":"2024-09-15T21:24:49.520416+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:49.520416+0000","last_undegraded":"2024-09-15T21:24:49.520416+0000","last_fullsized":"2024-09-15T21:24:49.520416+0000","mapping_epoch":49,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12395,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13787","reported_seq":14571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168661+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:54.168661+0000","last_peered":"2024-09-15T21:24:54.168661+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:54.168661+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13732","reported_seq":14157,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.685649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:49.685649+0000","last_peered":"2024-09-15T21:24:49.685649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:49.685649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12283,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1441}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13960","reported_seq":14370,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.758005+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:49.758005+0000","last_peered":"2024-09-15T21:24:49.758005+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:49.758005+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18792448,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14198","reported_seq":14941,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.117439+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:54.117439+0000","last_peered":"2024-09-15T21:24:54.117439+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:54.117439+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14018","reported_seq":19980,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168759+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:54.168759+0000","last_peered":"2024-09-15T21:24:54.168759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:54.168759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17997824,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14004","reported_seq":14662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.164301+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:54.164301+0000","last_peered":"2024-09-15T21:24:54.164301+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:54.164301+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14073","reported_seq":21167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.480159+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:49.480159+0000","last_peered":"2024-09-15T21:24:49.480159+0000","last_clean":"2024-09-15T21:24:49.480159+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:49.480159+0000","last_undegraded":"2024-09-15T21:24:49.480159+0000","last_fullsized":"2024-09-15T21:24:49.480159+0000","mapping_epoch":47,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18616937,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11733,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":294793857,"num_objects":71973,"num_object_clones":0,"num_object_copies":143946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71973,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195806,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15292,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":758804480,"data_stored":758740672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522500,"kb_used_data":256180,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93849340,"statfs":{"total":96636764160,"available":96101724160,"internally_reserved":0,"allocated":262328320,"data_stored":262037463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650200,"kb_used_data":267336,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93721640,"statfs":{"total":96636764160,"available":95970959360,"internally_reserved":0,"allocated":273752064,"data_stored":273465279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541428,"kb_used_data":220964,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93830412,"statfs":{"total":96636764160,"available":96082341888,"internally_reserved":0,"allocated":226267136,"data_stored":225931052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272343040,"data_stored":272339561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224780288,"data_stored":224727510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261681152,"data_stored":261673601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:55.104 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:55.399 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:55.399 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-15T21:24:54.476391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":294793876,"num_objects":71975,"num_object_clones":0,"num_object_copies":143950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195808,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15293,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160795,"ondisk_log_size":160795,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714128,"kb_used_data":744480,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":281401392,"statfs":{"total":289910292480,"available":288155025408,"internally_reserved":0,"allocated":762347520,"data_stored":761433794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":992857098},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,7,5,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23625728,"num_objects":-5768,"num_object_clones":0,"num_object_copies":-11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5768,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008896"},"pg_stats":[{"pgid":"2.f","version":"50'13997","reported_seq":14755,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.175903+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:54.175903+0000","last_peered":"2024-09-15T21:24:54.175903+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:54.175903+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493440,"num_objects":4515,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1484}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14061","reported_seq":14167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.524861+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:49.524861+0000","last_peered":"2024-09-15T21:24:49.524861+0000","last_clean":"2024-09-15T21:24:49.524861+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:49.524861+0000","last_undegraded":"2024-09-15T21:24:49.524861+0000","last_fullsized":"2024-09-15T21:24:49.524861+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18755584,"num_objects":4579,"num_object_clones":0,"num_object_copies":9158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12787,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14116","reported_seq":21040,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.440992+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:49.440992+0000","last_peered":"2024-09-15T21:24:49.440992+0000","last_clean":"2024-09-15T21:24:49.440992+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:49.440992+0000","last_undegraded":"2024-09-15T21:24:49.440992+0000","last_fullsized":"2024-09-15T21:24:49.440992+0000","mapping_epoch":45,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19468312,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14028","reported_seq":14107,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.732692+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:49.732692+0000","last_peered":"2024-09-15T21:24:49.732692+0000","last_clean":"2024-09-15T21:24:49.732692+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:49.732692+0000","last_undegraded":"2024-09-15T21:24:49.732692+0000","last_fullsized":"2024-09-15T21:24:49.732692+0000","mapping_epoch":36,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14254","reported_seq":14382,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.198509+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:54.198509+0000","last_peered":"2024-09-15T21:24:54.198509+0000","last_clean":"2024-09-15T21:24:54.198509+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:54.198509+0000","last_undegraded":"2024-09-15T21:24:54.198509+0000","last_fullsized":"2024-09-15T21:24:54.198509+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14019","reported_seq":14842,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.780696+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:53.780696+0000","last_peered":"2024-09-15T21:24:53.780696+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:53.780696+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14360","reported_seq":15105,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.143949+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:54.143949+0000","last_peered":"2024-09-15T21:24:54.143949+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:54.143949+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11996,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13976","reported_seq":14105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.121561+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:54.121561+0000","last_peered":"2024-09-15T21:24:54.121561+0000","last_clean":"2024-09-15T21:24:54.121561+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:54.121561+0000","last_undegraded":"2024-09-15T21:24:54.121561+0000","last_fullsized":"2024-09-15T21:24:54.121561+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13809","reported_seq":20631,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.520416+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:49.520416+0000","last_peered":"2024-09-15T21:24:49.520416+0000","last_clean":"2024-09-15T21:24:49.520416+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:49.520416+0000","last_undegraded":"2024-09-15T21:24:49.520416+0000","last_fullsized":"2024-09-15T21:24:49.520416+0000","mapping_epoch":49,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12395,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13787","reported_seq":14571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168661+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:54.168661+0000","last_peered":"2024-09-15T21:24:54.168661+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:54.168661+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13732","reported_seq":14157,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.685649+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:49.685649+0000","last_peered":"2024-09-15T21:24:49.685649+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:49.685649+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12283,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1441}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":88,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659438+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659438+0000","last_peered":"2024-09-15T21:23:34.659438+0000","last_clean":"2024-09-15T21:23:34.659438+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659438+0000","last_undegraded":"2024-09-15T21:23:34.659438+0000","last_fullsized":"2024-09-15T21:23:34.659438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'13960","reported_seq":14370,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:49.758005+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:49.758005+0000","last_peered":"2024-09-15T21:24:49.758005+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:49.758005+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18792448,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12359,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14198","reported_seq":14941,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.117439+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:54.117439+0000","last_peered":"2024-09-15T21:24:54.117439+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:54.117439+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659249+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659249+0000","last_peered":"2024-09-15T21:23:34.659249+0000","last_clean":"2024-09-15T21:23:34.659249+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659249+0000","last_undegraded":"2024-09-15T21:23:34.659249+0000","last_fullsized":"2024-09-15T21:23:34.659249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14018","reported_seq":19980,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168759+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:54.168759+0000","last_peered":"2024-09-15T21:24:54.168759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:54.168759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17997824,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14004","reported_seq":14662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.164301+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:54.164301+0000","last_peered":"2024-09-15T21:24:54.164301+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:54.164301+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14073","reported_seq":21167,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:49.480159+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:49.480159+0000","last_peered":"2024-09-15T21:24:49.480159+0000","last_clean":"2024-09-15T21:24:49.480159+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:49.480159+0000","last_undegraded":"2024-09-15T21:24:49.480159+0000","last_fullsized":"2024-09-15T21:24:49.480159+0000","mapping_epoch":47,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18616937,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11733,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":294793857,"num_objects":71973,"num_object_clones":0,"num_object_copies":143946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71973,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195806,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15292,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":758804480,"data_stored":758740672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953564,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522500,"kb_used_data":256180,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93849340,"statfs":{"total":96636764160,"available":96101724160,"internally_reserved":0,"allocated":262328320,"data_stored":262037463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650200,"kb_used_data":267336,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93721640,"statfs":{"total":96636764160,"available":95970959360,"internally_reserved":0,"allocated":273752064,"data_stored":273465279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541428,"kb_used_data":220964,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93830412,"statfs":{"total":96636764160,"available":96082341888,"internally_reserved":0,"allocated":226267136,"data_stored":225931052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":328134316},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272343040,"data_stored":272339561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224780288,"data_stored":224727510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261681152,"data_stored":261673601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:55.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:55.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'13997", 'reported_seq': 14755, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.175903+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:54.175903+0000', 'last_peered': '2024-09-15T21:24:54.175903+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:54.175903+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18493440, 'num_objects': 4515, 'num_object_clones': 0, 'num_object_copies': 9030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13997, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1484}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:55.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'14019", 'reported_seq': 14842, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:53.780696+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:53.780696+0000', 'last_peered': '2024-09-15T21:24:53.780696+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:53.780696+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17866752, 'num_objects': 4362, 'num_object_clones': 0, 'num_object_copies': 8724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11444, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'14360", 'reported_seq': 15105, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.143949+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:54.143949+0000', 'last_peered': '2024-09-15T21:24:54.143949+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:54.143949+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18612224, 'num_objects': 4544, 'num_object_clones': 0, 'num_object_copies': 9088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11996, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'13787", 'reported_seq': 14571, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.168661+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:54.168661+0000', 'last_peered': '2024-09-15T21:24:54.168661+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:54.168661+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17395712, 'num_objects': 4247, 'num_object_clones': 0, 'num_object_copies': 8494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11313, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1232}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'13732", 'reported_seq': 14157, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:49.685649+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:49.685649+0000', 'last_peered': '2024-09-15T21:24:49.685649+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:49.685649+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'3700", 'ondisk_log_start': "44'3700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18649088, 'num_objects': 4553, 'num_object_clones': 0, 'num_object_copies': 9106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12283, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1441}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'13960", 'reported_seq': 14370, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:49.758005+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:49.758005+0000', 'last_peered': '2024-09-15T21:24:49.758005+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:49.758005+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18792448, 'num_objects': 4588, 'num_object_clones': 0, 'num_object_copies': 9176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12359, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1382}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:55.418 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'14198", 'reported_seq': 14941, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.117439+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:54.117439+0000', 'last_peered': '2024-09-15T21:24:54.117439+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:54.117439+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18481152, 'num_objects': 4512, 'num_object_clones': 0, 'num_object_copies': 9024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11505, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:55.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'14018", 'reported_seq': 19980, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.168759+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:54.168759+0000', 'last_peered': '2024-09-15T21:24:54.168759+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:54.168759+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17997824, 'num_objects': 4394, 'num_object_clones': 0, 'num_object_copies': 8788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 471, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4394, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11603, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2911, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:55.419 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'14004", 'reported_seq': 14662, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:54.164301+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:54.164301+0000', 'last_peered': '2024-09-15T21:24:54.164301+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:54.164301+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17915904, 'num_objects': 4374, 'num_object_clones': 0, 'num_object_copies': 8748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4374, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11651, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:55.419 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:55.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:55.917 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25273/143950 objects degraded (17.557%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":508,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71975,"data_bytes":294793876,"bytes_used":1755267072,"bytes_avail":288155025408,"bytes_total":289910292480,"degraded_objects":25273,"degraded_total":143950,"degraded_ratio":0.17556790552275095,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":687,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":480},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:24:55.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:55.957 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:56.613 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:24:56.614 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:24:56.675 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-15T21:24:56.476826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290898580,"num_objects":71024,"num_object_clones":0,"num_object_copies":142048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196759,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15293,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715732,"kb_used_data":733604,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281399788,"statfs":{"total":289910292480,"available":288153382912,"internally_reserved":0,"allocated":751210496,"data_stored":750296770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1005636618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,3,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22183936,"num_objects":-5416,"num_object_clones":0,"num_object_copies":-10832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5416,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004409"},"pg_stats":[{"pgid":"2.f","version":"50'13997","reported_seq":14755,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.175903+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:54.175903+0000","last_peered":"2024-09-15T21:24:54.175903+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:54.175903+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493440,"num_objects":4515,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1484}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14206","reported_seq":14312,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.764856+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:54.764856+0000","last_peered":"2024-09-15T21:24:54.764856+0000","last_clean":"2024-09-15T21:24:54.764856+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:54.764856+0000","last_undegraded":"2024-09-15T21:24:54.764856+0000","last_fullsized":"2024-09-15T21:24:54.764856+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18161664,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14263","reported_seq":21187,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.571607+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:54.571607+0000","last_peered":"2024-09-15T21:24:54.571607+0000","last_clean":"2024-09-15T21:24:54.571607+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:54.571607+0000","last_undegraded":"2024-09-15T21:24:54.571607+0000","last_fullsized":"2024-09-15T21:24:54.571607+0000","mapping_epoch":45,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18866200,"num_objects":4607,"num_object_clones":0,"num_object_copies":9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4607,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12986,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14164","reported_seq":14243,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.757774+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:54.757774+0000","last_peered":"2024-09-15T21:24:54.757774+0000","last_clean":"2024-09-15T21:24:54.757774+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:54.757774+0000","last_undegraded":"2024-09-15T21:24:54.757774+0000","last_fullsized":"2024-09-15T21:24:54.757774+0000","mapping_epoch":36,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14164,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14254","reported_seq":14382,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.198509+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:54.198509+0000","last_peered":"2024-09-15T21:24:54.198509+0000","last_clean":"2024-09-15T21:24:54.198509+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:54.198509+0000","last_undegraded":"2024-09-15T21:24:54.198509+0000","last_fullsized":"2024-09-15T21:24:54.198509+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14019","reported_seq":14842,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.780696+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:53.780696+0000","last_peered":"2024-09-15T21:24:53.780696+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:53.780696+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14360","reported_seq":15105,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.143949+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:54.143949+0000","last_peered":"2024-09-15T21:24:54.143949+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:54.143949+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11996,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13976","reported_seq":14105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.121561+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:54.121561+0000","last_peered":"2024-09-15T21:24:54.121561+0000","last_clean":"2024-09-15T21:24:54.121561+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:54.121561+0000","last_undegraded":"2024-09-15T21:24:54.121561+0000","last_fullsized":"2024-09-15T21:24:54.121561+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13943","reported_seq":20766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.489020+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:54.489020+0000","last_peered":"2024-09-15T21:24:54.489020+0000","last_clean":"2024-09-15T21:24:54.489020+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:54.489020+0000","last_undegraded":"2024-09-15T21:24:54.489020+0000","last_fullsized":"2024-09-15T21:24:54.489020+0000","mapping_epoch":49,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12529,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13787","reported_seq":14571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168661+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:54.168661+0000","last_peered":"2024-09-15T21:24:54.168661+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:54.168661+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13866","reported_seq":14293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.780041+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:54.780041+0000","last_peered":"2024-09-15T21:24:54.780041+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:54.780041+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12417,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659481+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659481+0000","last_peered":"2024-09-15T21:23:34.659481+0000","last_clean":"2024-09-15T21:23:34.659481+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659481+0000","last_undegraded":"2024-09-15T21:23:34.659481+0000","last_fullsized":"2024-09-15T21:23:34.659481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'14085","reported_seq":14497,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.713441+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:54.713441+0000","last_peered":"2024-09-15T21:24:54.713441+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:54.713441+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18280448,"num_objects":4463,"num_object_clones":0,"num_object_copies":8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12484,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1257}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14198","reported_seq":14941,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.117439+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:54.117439+0000","last_peered":"2024-09-15T21:24:54.117439+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:54.117439+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659287+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659287+0000","last_peered":"2024-09-15T21:23:34.659287+0000","last_clean":"2024-09-15T21:23:34.659287+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659287+0000","last_undegraded":"2024-09-15T21:23:34.659287+0000","last_fullsized":"2024-09-15T21:23:34.659287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14018","reported_seq":19980,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168759+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:54.168759+0000","last_peered":"2024-09-15T21:24:54.168759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:54.168759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17997824,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14004","reported_seq":14662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.164301+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:54.164301+0000","last_peered":"2024-09-15T21:24:54.164301+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:54.164301+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14203","reported_seq":21297,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.488626+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:54.488626+0000","last_peered":"2024-09-15T21:24:54.488626+0000","last_clean":"2024-09-15T21:24:54.488626+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:54.488626+0000","last_undegraded":"2024-09-15T21:24:54.488626+0000","last_fullsized":"2024-09-15T21:24:54.488626+0000","mapping_epoch":47,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18084457,"num_objects":4416,"num_object_clones":0,"num_object_copies":8832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290898561,"num_objects":71022,"num_object_clones":0,"num_object_copies":142044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196757,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15292,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":747622400,"data_stored":747558592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522376,"kb_used_data":251896,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93849464,"statfs":{"total":96636764160,"available":96101851136,"internally_reserved":0,"allocated":257941504,"data_stored":257650647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650200,"kb_used_data":267336,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93721640,"statfs":{"total":96636764160,"available":95970959360,"internally_reserved":0,"allocated":273752064,"data_stored":273465279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543156,"kb_used_data":214372,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93828684,"statfs":{"total":96636764160,"available":96080572416,"internally_reserved":0,"allocated":219516928,"data_stored":219180844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272343040,"data_stored":272339561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217989120,"data_stored":217936342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257290240,"data_stored":257282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:59.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:00.426 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:00.428 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:00.485 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-15T21:24:58.477333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290898580,"num_objects":71024,"num_object_clones":0,"num_object_copies":142048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71024,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196759,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15293,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715732,"kb_used_data":733604,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281399788,"statfs":{"total":289910292480,"available":288153382912,"internally_reserved":0,"allocated":751210496,"data_stored":750296770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1005636618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,3,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18051072,"num_objects":-4407,"num_object_clones":0,"num_object_copies":-8814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4407,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"2.f","version":"50'13997","reported_seq":14755,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.175903+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:54.175903+0000","last_peered":"2024-09-15T21:24:54.175903+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:54.175903+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493440,"num_objects":4515,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1484}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14206","reported_seq":14312,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.764856+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:54.764856+0000","last_peered":"2024-09-15T21:24:54.764856+0000","last_clean":"2024-09-15T21:24:54.764856+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:54.764856+0000","last_undegraded":"2024-09-15T21:24:54.764856+0000","last_fullsized":"2024-09-15T21:24:54.764856+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18161664,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14263","reported_seq":21187,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.571607+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:54.571607+0000","last_peered":"2024-09-15T21:24:54.571607+0000","last_clean":"2024-09-15T21:24:54.571607+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:54.571607+0000","last_undegraded":"2024-09-15T21:24:54.571607+0000","last_fullsized":"2024-09-15T21:24:54.571607+0000","mapping_epoch":45,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18866200,"num_objects":4607,"num_object_clones":0,"num_object_copies":9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4607,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12986,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14164","reported_seq":14243,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.757774+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:54.757774+0000","last_peered":"2024-09-15T21:24:54.757774+0000","last_clean":"2024-09-15T21:24:54.757774+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:54.757774+0000","last_undegraded":"2024-09-15T21:24:54.757774+0000","last_fullsized":"2024-09-15T21:24:54.757774+0000","mapping_epoch":36,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14164,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14254","reported_seq":14382,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.198509+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:54.198509+0000","last_peered":"2024-09-15T21:24:54.198509+0000","last_clean":"2024-09-15T21:24:54.198509+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:54.198509+0000","last_undegraded":"2024-09-15T21:24:54.198509+0000","last_fullsized":"2024-09-15T21:24:54.198509+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14019","reported_seq":14842,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.780696+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:53.780696+0000","last_peered":"2024-09-15T21:24:53.780696+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:53.780696+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11444,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14360","reported_seq":15105,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.143949+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:54.143949+0000","last_peered":"2024-09-15T21:24:54.143949+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:54.143949+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18612224,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11996,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'13976","reported_seq":14105,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.121561+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:54.121561+0000","last_peered":"2024-09-15T21:24:54.121561+0000","last_clean":"2024-09-15T21:24:54.121561+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:54.121561+0000","last_undegraded":"2024-09-15T21:24:54.121561+0000","last_fullsized":"2024-09-15T21:24:54.121561+0000","mapping_epoch":35,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'13943","reported_seq":20766,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.489020+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:54.489020+0000","last_peered":"2024-09-15T21:24:54.489020+0000","last_clean":"2024-09-15T21:24:54.489020+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:54.489020+0000","last_undegraded":"2024-09-15T21:24:54.489020+0000","last_fullsized":"2024-09-15T21:24:54.489020+0000","mapping_epoch":49,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12529,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13787","reported_seq":14571,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168661+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:54.168661+0000","last_peered":"2024-09-15T21:24:54.168661+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:54.168661+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3700","ondisk_log_start":"44'3700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17395712,"num_objects":4247,"num_object_clones":0,"num_object_copies":8494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'13866","reported_seq":14293,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.780041+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:54.780041+0000","last_peered":"2024-09-15T21:24:54.780041+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:54.780041+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'3800","ondisk_log_start":"44'3800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12417,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659481+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659481+0000","last_peered":"2024-09-15T21:23:34.659481+0000","last_clean":"2024-09-15T21:23:34.659481+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659481+0000","last_undegraded":"2024-09-15T21:23:34.659481+0000","last_fullsized":"2024-09-15T21:23:34.659481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'14085","reported_seq":14497,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.713441+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:54.713441+0000","last_peered":"2024-09-15T21:24:54.713441+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:54.713441+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18280448,"num_objects":4463,"num_object_clones":0,"num_object_copies":8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12484,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1257}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14198","reported_seq":14941,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.117439+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:54.117439+0000","last_peered":"2024-09-15T21:24:54.117439+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:54.117439+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659287+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659287+0000","last_peered":"2024-09-15T21:23:34.659287+0000","last_clean":"2024-09-15T21:23:34.659287+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659287+0000","last_undegraded":"2024-09-15T21:23:34.659287+0000","last_fullsized":"2024-09-15T21:23:34.659287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14018","reported_seq":19980,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.168759+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:54.168759+0000","last_peered":"2024-09-15T21:24:54.168759+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:54.168759+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17997824,"num_objects":4394,"num_object_clones":0,"num_object_copies":8788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14004","reported_seq":14662,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:54.164301+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:54.164301+0000","last_peered":"2024-09-15T21:24:54.164301+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:54.164301+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.661963+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.661963+0000","last_peered":"2024-09-15T21:23:34.661963+0000","last_clean":"2024-09-15T21:23:34.661963+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.661963+0000","last_undegraded":"2024-09-15T21:23:34.661963+0000","last_fullsized":"2024-09-15T21:23:34.661963+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14203","reported_seq":21297,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:54.488626+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:54.488626+0000","last_peered":"2024-09-15T21:24:54.488626+0000","last_clean":"2024-09-15T21:24:54.488626+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:54.488626+0000","last_undegraded":"2024-09-15T21:24:54.488626+0000","last_fullsized":"2024-09-15T21:24:54.488626+0000","mapping_epoch":47,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18084457,"num_objects":4416,"num_object_clones":0,"num_object_copies":8832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290898561,"num_objects":71022,"num_object_clones":0,"num_object_copies":142044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196757,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15292,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":747622400,"data_stored":747558592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953565,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522376,"kb_used_data":251896,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93849464,"statfs":{"total":96636764160,"available":96101851136,"internally_reserved":0,"allocated":257941504,"data_stored":257650647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149792,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":650200,"kb_used_data":267336,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93721640,"statfs":{"total":96636764160,"available":95970959360,"internally_reserved":0,"allocated":273752064,"data_stored":273465279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":392031921},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543156,"kb_used_data":214372,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93828684,"statfs":{"total":96636764160,"available":96080572416,"internally_reserved":0,"allocated":219516928,"data_stored":219180844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":336653996},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272343040,"data_stored":272339561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217989120,"data_stored":217936342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257290240,"data_stored":257282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:00.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:01.158 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:01.161 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:01.209 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-15T21:25:00.478197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":282911380,"num_objects":69074,"num_object_clones":0,"num_object_copies":138148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198709,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15649,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160596,"ondisk_log_size":160596,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700824,"kb_used_data":717276,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281414696,"statfs":{"total":289910292480,"available":288168648704,"internally_reserved":0,"allocated":734490624,"data_stored":733576898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,7,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26038272,"num_objects":-6357,"num_object_clones":0,"num_object_copies":-12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6357,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003228"},"pg_stats":[{"pgid":"2.f","version":"50'14135","reported_seq":14889,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.221783+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:59.221783+0000","last_peered":"2024-09-15T21:24:59.221783+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:59.221783+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14327","reported_seq":14433,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.812581+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:59.812581+0000","last_peered":"2024-09-15T21:24:59.812581+0000","last_clean":"2024-09-15T21:24:59.812581+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:59.812581+0000","last_undegraded":"2024-09-15T21:24:59.812581+0000","last_fullsized":"2024-09-15T21:24:59.812581+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13053,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14381","reported_seq":21306,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.567913+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:59.567913+0000","last_peered":"2024-09-15T21:24:59.567913+0000","last_clean":"2024-09-15T21:24:59.567913+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:59.567913+0000","last_undegraded":"2024-09-15T21:24:59.567913+0000","last_fullsized":"2024-09-15T21:24:59.567913+0000","mapping_epoch":45,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18382872,"num_objects":4489,"num_object_clones":0,"num_object_copies":8978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13104,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14280","reported_seq":14359,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.784799+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:59.784799+0000","last_peered":"2024-09-15T21:24:59.784799+0000","last_clean":"2024-09-15T21:24:59.784799+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:59.784799+0000","last_undegraded":"2024-09-15T21:24:59.784799+0000","last_fullsized":"2024-09-15T21:24:59.784799+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17571840,"num_objects":4290,"num_object_clones":0,"num_object_copies":8580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14356","reported_seq":14484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.097100+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:59.097100+0000","last_peered":"2024-09-15T21:24:59.097100+0000","last_clean":"2024-09-15T21:24:59.097100+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:59.097100+0000","last_undegraded":"2024-09-15T21:24:59.097100+0000","last_fullsized":"2024-09-15T21:24:59.097100+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17887232,"num_objects":4367,"num_object_clones":0,"num_object_copies":8734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12214,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14117","reported_seq":14940,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.120167+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:59.120167+0000","last_peered":"2024-09-15T21:24:59.120167+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:59.120167+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17465344,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11542,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14498","reported_seq":15244,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.207319+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:59.207319+0000","last_peered":"2024-09-15T21:24:59.207319+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:59.207319+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12134,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'14101","reported_seq":14230,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.164013+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:59.164013+0000","last_peered":"2024-09-15T21:24:59.164013+0000","last_clean":"2024-09-15T21:24:59.164013+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:59.164013+0000","last_undegraded":"2024-09-15T21:24:59.164013+0000","last_fullsized":"2024-09-15T21:24:59.164013+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'14072","reported_seq":20895,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.622671+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:59.622671+0000","last_peered":"2024-09-15T21:24:59.622671+0000","last_clean":"2024-09-15T21:24:59.622671+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:59.622671+0000","last_undegraded":"2024-09-15T21:24:59.622671+0000","last_fullsized":"2024-09-15T21:24:59.622671+0000","mapping_epoch":49,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13906","reported_seq":14690,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.189428+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:59.189428+0000","last_peered":"2024-09-15T21:24:59.189428+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:59.189428+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16908288,"num_objects":4128,"num_object_clones":0,"num_object_copies":8256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11432,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'14005","reported_seq":14432,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.756584+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:59.756584+0000","last_peered":"2024-09-15T21:24:59.756584+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:59.756584+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17530880,"num_objects":4280,"num_object_clones":0,"num_object_copies":8560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1168}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659481+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659481+0000","last_peered":"2024-09-15T21:23:34.659481+0000","last_clean":"2024-09-15T21:23:34.659481+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659481+0000","last_undegraded":"2024-09-15T21:23:34.659481+0000","last_fullsized":"2024-09-15T21:23:34.659481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'14202","reported_seq":14614,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.741437+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:59.741437+0000","last_peered":"2024-09-15T21:24:59.741437+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:59.741437+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14342","reported_seq":15081,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.173546+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:59.173546+0000","last_peered":"2024-09-15T21:24:59.173546+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:59.173546+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11649,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659287+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659287+0000","last_peered":"2024-09-15T21:23:34.659287+0000","last_clean":"2024-09-15T21:23:34.659287+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659287+0000","last_undegraded":"2024-09-15T21:23:34.659287+0000","last_fullsized":"2024-09-15T21:23:34.659287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14133","reported_seq":20807,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.205931+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:59.205931+0000","last_peered":"2024-09-15T21:24:59.205931+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:59.205931+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11718,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3267,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14121","reported_seq":14780,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.189528+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:59.189528+0000","last_peered":"2024-09-15T21:24:59.189528+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:59.189528+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17436672,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11768,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.662014+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.662014+0000","last_peered":"2024-09-15T21:23:34.662014+0000","last_clean":"2024-09-15T21:23:34.662014+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.662014+0000","last_undegraded":"2024-09-15T21:23:34.662014+0000","last_fullsized":"2024-09-15T21:23:34.662014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14317","reported_seq":21411,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.420453+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:59.420453+0000","last_peered":"2024-09-15T21:24:59.420453+0000","last_clean":"2024-09-15T21:24:59.420453+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:59.420453+0000","last_undegraded":"2024-09-15T21:24:59.420453+0000","last_fullsized":"2024-09-15T21:24:59.420453+0000","mapping_epoch":47,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17617513,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11977,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":282911361,"num_objects":69072,"num_object_clones":0,"num_object_copies":138144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198707,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15648,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":731623424,"data_stored":731559616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160593,"ondisk_log_size":160593,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953566,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518580,"kb_used_data":248096,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93853260,"statfs":{"total":96636764160,"available":96105738240,"internally_reserved":0,"allocated":254050304,"data_stored":253759447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636272,"kb_used_data":261728,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93735568,"statfs":{"total":96636764160,"available":95985221632,"internally_reserved":0,"allocated":268009472,"data_stored":267722687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545972,"kb_used_data":207452,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93825868,"statfs":{"total":96636764160,"available":96077688832,"internally_reserved":0,"allocated":212430848,"data_stored":212094764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266440704,"data_stored":266437225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":211918848,"data_stored":211866070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253263872,"data_stored":253256321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:01.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:01.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "50'14135", 'reported_seq': 14889, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.221783+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:24:59.221783+0000', 'last_peered': '2024-09-15T21:24:59.221783+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:24:59.221783+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17928192, 'num_objects': 4377, 'num_object_clones': 0, 'num_object_copies': 8754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14135, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1346}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "50'14117", 'reported_seq': 14940, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.120167+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:24:59.120167+0000', 'last_peered': '2024-09-15T21:24:59.120167+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:24:59.120167+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17465344, 'num_objects': 4264, 'num_object_clones': 0, 'num_object_copies': 8528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11542, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1235}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "50'14498", 'reported_seq': 15244, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.207319+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:24:59.207319+0000', 'last_peered': '2024-09-15T21:24:59.207319+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:24:59.207319+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18046976, 'num_objects': 4406, 'num_object_clones': 0, 'num_object_copies': 8812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12134, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "50'13906", 'reported_seq': 14690, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.189428+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:24:59.189428+0000', 'last_peered': '2024-09-15T21:24:59.189428+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:24:59.189428+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'3900", 'ondisk_log_start': "44'3900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16908288, 'num_objects': 4128, 'num_object_clones': 0, 'num_object_copies': 8256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11432, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1113}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "50'14005", 'reported_seq': 14432, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.756584+0000', 'last_change': '2024-09-15T21:18:43.842340+0000', 'last_active': '2024-09-15T21:24:59.756584+0000', 'last_peered': '2024-09-15T21:24:59.756584+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:24:59.756584+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17530880, 'num_objects': 4280, 'num_object_clones': 0, 'num_object_copies': 8560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12556, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1168}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "50'14202", 'reported_seq': 14614, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.741437+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:24:59.741437+0000', 'last_peered': '2024-09-15T21:24:59.741437+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:24:59.741437+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17801216, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12601, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1140}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:01.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:01.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "50'14342", 'reported_seq': 15081, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.173546+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:24:59.173546+0000', 'last_peered': '2024-09-15T21:24:59.173546+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:24:59.173546+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17891328, 'num_objects': 4368, 'num_object_clones': 0, 'num_object_copies': 8736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4368, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11649, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1238}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:01.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "50'14133", 'reported_seq': 20807, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.205931+0000', 'last_change': '2024-09-15T21:23:33.381118+0000', 'last_active': '2024-09-15T21:24:59.205931+0000', 'last_peered': '2024-09-15T21:24:59.205931+0000', 'last_clean': '2024-09-15T21:18:41.339426+0000', 'last_became_active': '2024-09-15T21:18:43.823629+0000', 'last_became_peered': '2024-09-15T21:18:43.823629+0000', 'last_unstale': '2024-09-15T21:24:59.205931+0000', 'last_undegraded': '2024-09-15T21:18:43.488473+0000', 'last_fullsized': '2024-09-15T21:18:43.370742+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2446", 'last_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:23.510869+0000', 'objects_scrubbed': 2407, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 755, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17526784, 'num_objects': 4279, 'num_object_clones': 0, 'num_object_copies': 8558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 115, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11718, 'num_write_kb': 27180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3267, 'num_bytes_recovered': 3321856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:01.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "50'14121", 'reported_seq': 14780, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:59.189528+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:24:59.189528+0000', 'last_peered': '2024-09-15T21:24:59.189528+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:24:59.189528+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17436672, 'num_objects': 4257, 'num_object_clones': 0, 'num_object_copies': 8514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11768, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:01.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:02.085 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:02.085 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24917/138148 objects degraded (18.036%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":515,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69074,"data_bytes":282911380,"bytes_used":1741643776,"bytes_avail":288168648704,"bytes_total":289910292480,"degraded_objects":24917,"degraded_total":138148,"degraded_ratio":0.18036453658395341,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":529},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:02.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:02.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:03.018 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-15T21:25:02.708946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":282911380,"num_objects":69074,"num_object_clones":0,"num_object_copies":138148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198709,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15649,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160596,"ondisk_log_size":160596,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700824,"kb_used_data":717276,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281414696,"statfs":{"total":289910292480,"available":288168648704,"internally_reserved":0,"allocated":734490624,"data_stored":733576898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,7,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16969728,"num_objects":-4143,"num_object_clones":0,"num_object_copies":-8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4143,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.232994"},"pg_stats":[{"pgid":"2.f","version":"50'14135","reported_seq":14889,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.221783+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:24:59.221783+0000","last_peered":"2024-09-15T21:24:59.221783+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:24:59.221783+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17928192,"num_objects":4377,"num_object_clones":0,"num_object_copies":8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"50'14327","reported_seq":14433,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.812581+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:24:59.812581+0000","last_peered":"2024-09-15T21:24:59.812581+0000","last_clean":"2024-09-15T21:24:59.812581+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:24:59.812581+0000","last_undegraded":"2024-09-15T21:24:59.812581+0000","last_fullsized":"2024-09-15T21:24:59.812581+0000","mapping_epoch":36,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13053,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"50'14381","reported_seq":21306,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.567913+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:24:59.567913+0000","last_peered":"2024-09-15T21:24:59.567913+0000","last_clean":"2024-09-15T21:24:59.567913+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:24:59.567913+0000","last_undegraded":"2024-09-15T21:24:59.567913+0000","last_fullsized":"2024-09-15T21:24:59.567913+0000","mapping_epoch":45,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18382872,"num_objects":4489,"num_object_clones":0,"num_object_copies":8978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4489,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13104,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"50'14280","reported_seq":14359,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.784799+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:24:59.784799+0000","last_peered":"2024-09-15T21:24:59.784799+0000","last_clean":"2024-09-15T21:24:59.784799+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:24:59.784799+0000","last_undegraded":"2024-09-15T21:24:59.784799+0000","last_fullsized":"2024-09-15T21:24:59.784799+0000","mapping_epoch":36,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17571840,"num_objects":4290,"num_object_clones":0,"num_object_copies":8580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"50'14356","reported_seq":14484,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.097100+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:24:59.097100+0000","last_peered":"2024-09-15T21:24:59.097100+0000","last_clean":"2024-09-15T21:24:59.097100+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:24:59.097100+0000","last_undegraded":"2024-09-15T21:24:59.097100+0000","last_fullsized":"2024-09-15T21:24:59.097100+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17887232,"num_objects":4367,"num_object_clones":0,"num_object_copies":8734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12214,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"50'14117","reported_seq":14940,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.120167+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:24:59.120167+0000","last_peered":"2024-09-15T21:24:59.120167+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:24:59.120167+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17465344,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11542,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"50'14498","reported_seq":15244,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.207319+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:24:59.207319+0000","last_peered":"2024-09-15T21:24:59.207319+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:24:59.207319+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18046976,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12134,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"50'14101","reported_seq":14230,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.164013+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:24:59.164013+0000","last_peered":"2024-09-15T21:24:59.164013+0000","last_clean":"2024-09-15T21:24:59.164013+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:24:59.164013+0000","last_undegraded":"2024-09-15T21:24:59.164013+0000","last_fullsized":"2024-09-15T21:24:59.164013+0000","mapping_epoch":35,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11886,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659156+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:23:34.659156+0000","last_peered":"2024-09-15T21:23:34.659156+0000","last_clean":"2024-09-15T21:23:34.659156+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:23:34.659156+0000","last_undegraded":"2024-09-15T21:23:34.659156+0000","last_fullsized":"2024-09-15T21:23:34.659156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"50'14072","reported_seq":20895,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.622671+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:24:59.622671+0000","last_peered":"2024-09-15T21:24:59.622671+0000","last_clean":"2024-09-15T21:24:59.622671+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:24:59.622671+0000","last_undegraded":"2024-09-15T21:24:59.622671+0000","last_fullsized":"2024-09-15T21:24:59.622671+0000","mapping_epoch":49,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17625088,"num_objects":4303,"num_object_clones":0,"num_object_copies":8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.658308+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:23:34.658308+0000","last_peered":"2024-09-15T21:23:34.658308+0000","last_clean":"2024-09-15T21:23:34.658308+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:23:34.658308+0000","last_undegraded":"2024-09-15T21:23:34.658308+0000","last_fullsized":"2024-09-15T21:23:34.658308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"50'13906","reported_seq":14690,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.189428+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:24:59.189428+0000","last_peered":"2024-09-15T21:24:59.189428+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:24:59.189428+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'3900","ondisk_log_start":"44'3900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16908288,"num_objects":4128,"num_object_clones":0,"num_object_copies":8256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11432,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":110,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665661+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:23:34.665661+0000","last_peered":"2024-09-15T21:23:34.665661+0000","last_clean":"2024-09-15T21:23:34.665661+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:23:34.665661+0000","last_undegraded":"2024-09-15T21:23:34.665661+0000","last_fullsized":"2024-09-15T21:23:34.665661+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"50'14005","reported_seq":14432,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.756584+0000","last_change":"2024-09-15T21:18:43.842340+0000","last_active":"2024-09-15T21:24:59.756584+0000","last_peered":"2024-09-15T21:24:59.756584+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:24:59.756584+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17530880,"num_objects":4280,"num_object_clones":0,"num_object_copies":8560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1168}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":89,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659481+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:23:34.659481+0000","last_peered":"2024-09-15T21:23:34.659481+0000","last_clean":"2024-09-15T21:23:34.659481+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:23:34.659481+0000","last_undegraded":"2024-09-15T21:23:34.659481+0000","last_fullsized":"2024-09-15T21:23:34.659481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"50'14202","reported_seq":14614,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.741437+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:24:59.741437+0000","last_peered":"2024-09-15T21:24:59.741437+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:24:59.741437+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1140}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"50'14342","reported_seq":15081,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.173546+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:24:59.173546+0000","last_peered":"2024-09-15T21:24:59.173546+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:24:59.173546+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11649,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659287+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:23:34.659287+0000","last_peered":"2024-09-15T21:23:34.659287+0000","last_clean":"2024-09-15T21:23:34.659287+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:23:34.659287+0000","last_undegraded":"2024-09-15T21:23:34.659287+0000","last_fullsized":"2024-09-15T21:23:34.659287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"50'14133","reported_seq":20807,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.205931+0000","last_change":"2024-09-15T21:23:33.381118+0000","last_active":"2024-09-15T21:24:59.205931+0000","last_peered":"2024-09-15T21:24:59.205931+0000","last_clean":"2024-09-15T21:18:41.339426+0000","last_became_active":"2024-09-15T21:18:43.823629+0000","last_became_peered":"2024-09-15T21:18:43.823629+0000","last_unstale":"2024-09-15T21:24:59.205931+0000","last_undegraded":"2024-09-15T21:18:43.488473+0000","last_fullsized":"2024-09-15T21:18:43.370742+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11718,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3267,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.665534+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:23:34.665534+0000","last_peered":"2024-09-15T21:23:34.665534+0000","last_clean":"2024-09-15T21:23:34.665534+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:23:34.665534+0000","last_undegraded":"2024-09-15T21:23:34.665534+0000","last_fullsized":"2024-09-15T21:23:34.665534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"50'14121","reported_seq":14780,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:59.189528+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:24:59.189528+0000","last_peered":"2024-09-15T21:24:59.189528+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:24:59.189528+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17436672,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11768,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.662014+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:23:34.662014+0000","last_peered":"2024-09-15T21:23:34.662014+0000","last_clean":"2024-09-15T21:23:34.662014+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:23:34.662014+0000","last_undegraded":"2024-09-15T21:23:34.662014+0000","last_fullsized":"2024-09-15T21:23:34.662014+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.659323+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:23:34.659323+0000","last_peered":"2024-09-15T21:23:34.659323+0000","last_clean":"2024-09-15T21:23:34.659323+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:23:34.659323+0000","last_undegraded":"2024-09-15T21:23:34.659323+0000","last_fullsized":"2024-09-15T21:23:34.659323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"50'14317","reported_seq":21411,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:24:59.420453+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:24:59.420453+0000","last_peered":"2024-09-15T21:24:59.420453+0000","last_clean":"2024-09-15T21:24:59.420453+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:24:59.420453+0000","last_undegraded":"2024-09-15T21:24:59.420453+0000","last_fullsized":"2024-09-15T21:24:59.420453+0000","mapping_epoch":47,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17617513,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11977,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":282911361,"num_objects":69072,"num_object_clones":0,"num_object_copies":138144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198707,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15648,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":731623424,"data_stored":731559616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160593,"ondisk_log_size":160593,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953566,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518580,"kb_used_data":248096,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93853260,"statfs":{"total":96636764160,"available":96105738240,"internally_reserved":0,"allocated":254050304,"data_stored":253759447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149793,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636272,"kb_used_data":261728,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93735568,"statfs":{"total":96636764160,"available":95985221632,"internally_reserved":0,"allocated":268009472,"data_stored":267722687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626511,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545972,"kb_used_data":207452,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93825868,"statfs":{"total":96636764160,"available":96077688832,"internally_reserved":0,"allocated":212430848,"data_stored":212094764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266440704,"data_stored":266437225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":211918848,"data_stored":211866070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253263872,"data_stored":253256321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:06.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:06.810 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:06.810 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:06.843 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-15T21:25:06.479537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273355412,"num_objects":66741,"num_object_clones":0,"num_object_copies":133482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201042,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15821,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1699032,"kb_used_data":698840,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":281416488,"statfs":{"total":289910292480,"available":288170483712,"internally_reserved":0,"allocated":715612160,"data_stored":714679040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1024117770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,10,6,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17543168,"num_objects":-4283,"num_object_clones":0,"num_object_copies":-8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002203"},"pg_stats":[{"pgid":"2.f","version":"52'14278","reported_seq":15040,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.239878+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:04.239878+0000","last_peered":"2024-09-15T21:25:04.239878+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:04.239878+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14278,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14464","reported_seq":14574,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.764350+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:04.764350+0000","last_peered":"2024-09-15T21:25:04.764350+0000","last_clean":"2024-09-15T21:25:04.764350+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:04.764350+0000","last_undegraded":"2024-09-15T21:25:04.764350+0000","last_fullsized":"2024-09-15T21:25:04.764350+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17104896,"num_objects":4176,"num_object_clones":0,"num_object_copies":8352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14549","reported_seq":21478,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.620816+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:04.620816+0000","last_peered":"2024-09-15T21:25:04.620816+0000","last_clean":"2024-09-15T21:25:04.620816+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:04.620816+0000","last_undegraded":"2024-09-15T21:25:04.620816+0000","last_fullsized":"2024-09-15T21:25:04.620816+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694744,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13272,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14453","reported_seq":14536,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.704682+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:04.704682+0000","last_peered":"2024-09-15T21:25:04.704682+0000","last_clean":"2024-09-15T21:25:04.704682+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:04.704682+0000","last_undegraded":"2024-09-15T21:25:04.704682+0000","last_fullsized":"2024-09-15T21:25:04.704682+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863232,"num_objects":4117,"num_object_clones":0,"num_object_copies":8234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14453,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14506","reported_seq":14638,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.271467+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:04.271467+0000","last_peered":"2024-09-15T21:25:04.271467+0000","last_clean":"2024-09-15T21:25:04.271467+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:04.271467+0000","last_undegraded":"2024-09-15T21:25:04.271467+0000","last_fullsized":"2024-09-15T21:25:04.271467+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14268","reported_seq":15095,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.256699+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:04.256699+0000","last_peered":"2024-09-15T21:25:04.256699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:04.256699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14649","reported_seq":15400,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.190555+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:04.190555+0000","last_peered":"2024-09-15T21:25:04.190555+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:04.190555+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14227","reported_seq":14360,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.226645+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:04.226645+0000","last_peered":"2024-09-15T21:25:04.226645+0000","last_clean":"2024-09-15T21:25:04.226645+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:04.226645+0000","last_undegraded":"2024-09-15T21:25:04.226645+0000","last_fullsized":"2024-09-15T21:25:04.226645+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12012,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14212","reported_seq":21039,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666398+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:04.666398+0000","last_peered":"2024-09-15T21:25:04.666398+0000","last_clean":"2024-09-15T21:25:04.666398+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:04.666398+0000","last_undegraded":"2024-09-15T21:25:04.666398+0000","last_fullsized":"2024-09-15T21:25:04.666398+0000","mapping_epoch":49,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14049","reported_seq":14837,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.262809+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:04.262809+0000","last_peered":"2024-09-15T21:25:04.262809+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:04.262809+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14144","reported_seq":14690,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.811925+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:04.811925+0000","last_peered":"2024-09-15T21:25:04.811925+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:04.811925+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12695,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1084}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14335","reported_seq":14751,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.791922+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:04.791922+0000","last_peered":"2024-09-15T21:25:04.791922+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:04.791922+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256448,"num_objects":4213,"num_object_clones":0,"num_object_copies":8426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12734,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14500","reported_seq":15248,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.238681+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:04.238681+0000","last_peered":"2024-09-15T21:25:04.238681+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:04.238681+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244160,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14278","reported_seq":21195,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666064+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:04.666064+0000","last_peered":"2024-09-15T21:25:04.666064+0000","last_clean":"2024-09-15T21:25:04.666064+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:04.666064+0000","last_undegraded":"2024-09-15T21:25:04.666064+0000","last_fullsized":"2024-09-15T21:25:04.666064+0000","mapping_epoch":51,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14266","reported_seq":14929,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.228584+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:04.228584+0000","last_peered":"2024-09-15T21:25:04.228584+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:04.228584+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11913,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730193+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730193+0000","last_peered":"2024-09-15T21:25:03.730193+0000","last_clean":"2024-09-15T21:25:03.730193+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730193+0000","last_undegraded":"2024-09-15T21:25:03.730193+0000","last_fullsized":"2024-09-15T21:25:03.730193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14448","reported_seq":21546,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.686221+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:04.686221+0000","last_peered":"2024-09-15T21:25:04.686221+0000","last_clean":"2024-09-15T21:25:04.686221+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:04.686221+0000","last_undegraded":"2024-09-15T21:25:04.686221+0000","last_fullsized":"2024-09-15T21:25:04.686221+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080937,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273355393,"num_objects":66739,"num_object_clones":0,"num_object_copies":133478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201040,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15820,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712564736,"data_stored":712507886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160826,"ondisk_log_size":160826,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518168,"kb_used_data":243524,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93853672,"statfs":{"total":96636764160,"available":96106160128,"internally_reserved":0,"allocated":249368576,"data_stored":249067174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633476,"kb_used_data":254760,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93738364,"statfs":{"total":96636764160,"available":95988084736,"internally_reserved":0,"allocated":260874240,"data_stored":260582193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626512,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547388,"kb_used_data":200556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93824452,"statfs":{"total":96636764160,"available":96076238848,"internally_reserved":0,"allocated":205369344,"data_stored":205029673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259268608,"data_stored":259265129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":204771328,"data_stored":204725508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248524800,"data_stored":248517249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:06.844 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:07.637 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:07.727 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-15T21:25:06.479537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273355412,"num_objects":66741,"num_object_clones":0,"num_object_copies":133482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201042,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15821,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1699032,"kb_used_data":698840,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":281416488,"statfs":{"total":289910292480,"available":288170483712,"internally_reserved":0,"allocated":715612160,"data_stored":714679040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1024117770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,10,6,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17543168,"num_objects":-4283,"num_object_clones":0,"num_object_copies":-8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002203"},"pg_stats":[{"pgid":"2.f","version":"52'14278","reported_seq":15040,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.239878+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:04.239878+0000","last_peered":"2024-09-15T21:25:04.239878+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:04.239878+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14278,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14464","reported_seq":14574,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.764350+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:04.764350+0000","last_peered":"2024-09-15T21:25:04.764350+0000","last_clean":"2024-09-15T21:25:04.764350+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:04.764350+0000","last_undegraded":"2024-09-15T21:25:04.764350+0000","last_fullsized":"2024-09-15T21:25:04.764350+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17104896,"num_objects":4176,"num_object_clones":0,"num_object_copies":8352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14549","reported_seq":21478,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.620816+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:04.620816+0000","last_peered":"2024-09-15T21:25:04.620816+0000","last_clean":"2024-09-15T21:25:04.620816+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:04.620816+0000","last_undegraded":"2024-09-15T21:25:04.620816+0000","last_fullsized":"2024-09-15T21:25:04.620816+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694744,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13272,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14453","reported_seq":14536,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.704682+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:04.704682+0000","last_peered":"2024-09-15T21:25:04.704682+0000","last_clean":"2024-09-15T21:25:04.704682+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:04.704682+0000","last_undegraded":"2024-09-15T21:25:04.704682+0000","last_fullsized":"2024-09-15T21:25:04.704682+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863232,"num_objects":4117,"num_object_clones":0,"num_object_copies":8234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14453,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14506","reported_seq":14638,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.271467+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:04.271467+0000","last_peered":"2024-09-15T21:25:04.271467+0000","last_clean":"2024-09-15T21:25:04.271467+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:04.271467+0000","last_undegraded":"2024-09-15T21:25:04.271467+0000","last_fullsized":"2024-09-15T21:25:04.271467+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14268","reported_seq":15095,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.256699+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:04.256699+0000","last_peered":"2024-09-15T21:25:04.256699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:04.256699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14649","reported_seq":15400,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.190555+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:04.190555+0000","last_peered":"2024-09-15T21:25:04.190555+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:04.190555+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14227","reported_seq":14360,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.226645+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:04.226645+0000","last_peered":"2024-09-15T21:25:04.226645+0000","last_clean":"2024-09-15T21:25:04.226645+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:04.226645+0000","last_undegraded":"2024-09-15T21:25:04.226645+0000","last_fullsized":"2024-09-15T21:25:04.226645+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12012,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14212","reported_seq":21039,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666398+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:04.666398+0000","last_peered":"2024-09-15T21:25:04.666398+0000","last_clean":"2024-09-15T21:25:04.666398+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:04.666398+0000","last_undegraded":"2024-09-15T21:25:04.666398+0000","last_fullsized":"2024-09-15T21:25:04.666398+0000","mapping_epoch":49,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14049","reported_seq":14837,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.262809+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:04.262809+0000","last_peered":"2024-09-15T21:25:04.262809+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:04.262809+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14144","reported_seq":14690,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.811925+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:04.811925+0000","last_peered":"2024-09-15T21:25:04.811925+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:04.811925+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12695,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1084}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14335","reported_seq":14751,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.791922+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:04.791922+0000","last_peered":"2024-09-15T21:25:04.791922+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:04.791922+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256448,"num_objects":4213,"num_object_clones":0,"num_object_copies":8426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12734,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14500","reported_seq":15248,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.238681+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:04.238681+0000","last_peered":"2024-09-15T21:25:04.238681+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:04.238681+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244160,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14278","reported_seq":21195,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666064+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:04.666064+0000","last_peered":"2024-09-15T21:25:04.666064+0000","last_clean":"2024-09-15T21:25:04.666064+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:04.666064+0000","last_undegraded":"2024-09-15T21:25:04.666064+0000","last_fullsized":"2024-09-15T21:25:04.666064+0000","mapping_epoch":51,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14266","reported_seq":14929,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.228584+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:04.228584+0000","last_peered":"2024-09-15T21:25:04.228584+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:04.228584+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11913,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730193+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730193+0000","last_peered":"2024-09-15T21:25:03.730193+0000","last_clean":"2024-09-15T21:25:03.730193+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730193+0000","last_undegraded":"2024-09-15T21:25:03.730193+0000","last_fullsized":"2024-09-15T21:25:03.730193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14448","reported_seq":21546,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.686221+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:04.686221+0000","last_peered":"2024-09-15T21:25:04.686221+0000","last_clean":"2024-09-15T21:25:04.686221+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:04.686221+0000","last_undegraded":"2024-09-15T21:25:04.686221+0000","last_fullsized":"2024-09-15T21:25:04.686221+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080937,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273355393,"num_objects":66739,"num_object_clones":0,"num_object_copies":133478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201040,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15820,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712564736,"data_stored":712507886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160826,"ondisk_log_size":160826,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518168,"kb_used_data":243524,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93853672,"statfs":{"total":96636764160,"available":96106160128,"internally_reserved":0,"allocated":249368576,"data_stored":249067174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633476,"kb_used_data":254760,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93738364,"statfs":{"total":96636764160,"available":95988084736,"internally_reserved":0,"allocated":260874240,"data_stored":260582193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626512,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547388,"kb_used_data":200556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93824452,"statfs":{"total":96636764160,"available":96076238848,"internally_reserved":0,"allocated":205369344,"data_stored":205029673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259268608,"data_stored":259265129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":204771328,"data_stored":204725508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248524800,"data_stored":248517249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:07.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:07.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'14278", 'reported_seq': 15040, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.239878+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:04.239878+0000', 'last_peered': '2024-09-15T21:25:04.239878+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:04.239878+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17342464, 'num_objects': 4234, 'num_object_clones': 0, 'num_object_copies': 8468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14278, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1203}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'14268", 'reported_seq': 15095, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.256699+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:04.256699+0000', 'last_peered': '2024-09-15T21:25:04.256699+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:04.256699+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16846848, 'num_objects': 4113, 'num_object_clones': 0, 'num_object_copies': 8226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11693, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'14649", 'reported_seq': 15400, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.190555+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:04.190555+0000', 'last_peered': '2024-09-15T21:25:04.190555+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:04.190555+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17428480, 'num_objects': 4255, 'num_object_clones': 0, 'num_object_copies': 8510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12285, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14049", 'reported_seq': 14837, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.262809+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:04.262809+0000', 'last_peered': '2024-09-15T21:25:04.262809+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:04.262809+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4000", 'ondisk_log_start': "44'4000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16322560, 'num_objects': 3985, 'num_object_clones': 0, 'num_object_copies': 7970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11575, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 970}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'14144", 'reported_seq': 14690, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.811925+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:04.811925+0000', 'last_peered': '2024-09-15T21:25:04.811925+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:04.811925+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16961536, 'num_objects': 4141, 'num_object_clones': 0, 'num_object_copies': 8282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3057, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12695, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 211, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1084}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'14335", 'reported_seq': 14751, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.791922+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:04.791922+0000', 'last_peered': '2024-09-15T21:25:04.791922+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:04.791922+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17256448, 'num_objects': 4213, 'num_object_clones': 0, 'num_object_copies': 8426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4213, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12734, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1007}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:07.729 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'14500", 'reported_seq': 15248, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.238681+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:04.238681+0000', 'last_peered': '2024-09-15T21:25:04.238681+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:04.238681+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17244160, 'num_objects': 4210, 'num_object_clones': 0, 'num_object_copies': 8420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11807, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1080}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:07.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'14266", 'reported_seq': 14929, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:04.228584+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:04.228584+0000', 'last_peered': '2024-09-15T21:25:04.228584+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:04.228584+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16842752, 'num_objects': 4112, 'num_object_clones': 0, 'num_object_copies': 8224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11913, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:07.730 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:08.505 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24745/133482 objects degraded (18.538%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":521,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66741,"data_bytes":273355412,"bytes_used":1739808768,"bytes_avail":288170483712,"bytes_total":289910292480,"degraded_objects":24745,"degraded_total":133482,"degraded_ratio":0.18538080040754557,"recovering_objects_per_sec":65,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":528,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":535},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:08.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:08.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:09.238 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:09.239 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:09.259 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-15T21:25:08.480016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273355412,"num_objects":66741,"num_object_clones":0,"num_object_copies":133482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66741,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201042,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15821,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1699032,"kb_used_data":698840,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":281416488,"statfs":{"total":289910292480,"available":288170483712,"internally_reserved":0,"allocated":715612160,"data_stored":714679040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1024117770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,10,6,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9555968,"num_objects":-2333,"num_object_clones":0,"num_object_copies":-4666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2333,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001819"},"pg_stats":[{"pgid":"2.f","version":"52'14278","reported_seq":15040,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.239878+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:04.239878+0000","last_peered":"2024-09-15T21:25:04.239878+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:04.239878+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17342464,"num_objects":4234,"num_object_clones":0,"num_object_copies":8468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14278,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1203}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14464","reported_seq":14574,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.764350+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:04.764350+0000","last_peered":"2024-09-15T21:25:04.764350+0000","last_clean":"2024-09-15T21:25:04.764350+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:04.764350+0000","last_undegraded":"2024-09-15T21:25:04.764350+0000","last_fullsized":"2024-09-15T21:25:04.764350+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17104896,"num_objects":4176,"num_object_clones":0,"num_object_copies":8352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14549","reported_seq":21478,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.620816+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:04.620816+0000","last_peered":"2024-09-15T21:25:04.620816+0000","last_clean":"2024-09-15T21:25:04.620816+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:04.620816+0000","last_undegraded":"2024-09-15T21:25:04.620816+0000","last_fullsized":"2024-09-15T21:25:04.620816+0000","mapping_epoch":45,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694744,"num_objects":4321,"num_object_clones":0,"num_object_copies":8642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13272,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14453","reported_seq":14536,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.704682+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:04.704682+0000","last_peered":"2024-09-15T21:25:04.704682+0000","last_clean":"2024-09-15T21:25:04.704682+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:04.704682+0000","last_undegraded":"2024-09-15T21:25:04.704682+0000","last_fullsized":"2024-09-15T21:25:04.704682+0000","mapping_epoch":36,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863232,"num_objects":4117,"num_object_clones":0,"num_object_copies":8234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14453,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14506","reported_seq":14638,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.271467+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:04.271467+0000","last_peered":"2024-09-15T21:25:04.271467+0000","last_clean":"2024-09-15T21:25:04.271467+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:04.271467+0000","last_undegraded":"2024-09-15T21:25:04.271467+0000","last_fullsized":"2024-09-15T21:25:04.271467+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14268","reported_seq":15095,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.256699+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:04.256699+0000","last_peered":"2024-09-15T21:25:04.256699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:04.256699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11693,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14649","reported_seq":15400,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.190555+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:04.190555+0000","last_peered":"2024-09-15T21:25:04.190555+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:04.190555+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17428480,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14227","reported_seq":14360,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.226645+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:04.226645+0000","last_peered":"2024-09-15T21:25:04.226645+0000","last_clean":"2024-09-15T21:25:04.226645+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:04.226645+0000","last_undegraded":"2024-09-15T21:25:04.226645+0000","last_fullsized":"2024-09-15T21:25:04.226645+0000","mapping_epoch":35,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12012,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14212","reported_seq":21039,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666398+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:04.666398+0000","last_peered":"2024-09-15T21:25:04.666398+0000","last_clean":"2024-09-15T21:25:04.666398+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:04.666398+0000","last_undegraded":"2024-09-15T21:25:04.666398+0000","last_fullsized":"2024-09-15T21:25:04.666398+0000","mapping_epoch":49,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17051648,"num_objects":4163,"num_object_clones":0,"num_object_copies":8326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14049","reported_seq":14837,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.262809+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:04.262809+0000","last_peered":"2024-09-15T21:25:04.262809+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:04.262809+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4000","ondisk_log_start":"44'4000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16322560,"num_objects":3985,"num_object_clones":0,"num_object_copies":7970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14144","reported_seq":14690,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.811925+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:04.811925+0000","last_peered":"2024-09-15T21:25:04.811925+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:04.811925+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12695,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1084}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14335","reported_seq":14751,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.791922+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:04.791922+0000","last_peered":"2024-09-15T21:25:04.791922+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:04.791922+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17256448,"num_objects":4213,"num_object_clones":0,"num_object_copies":8426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12734,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14500","reported_seq":15248,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.238681+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:04.238681+0000","last_peered":"2024-09-15T21:25:04.238681+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:04.238681+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17244160,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14278","reported_seq":21195,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.666064+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:04.666064+0000","last_peered":"2024-09-15T21:25:04.666064+0000","last_clean":"2024-09-15T21:25:04.666064+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:04.666064+0000","last_undegraded":"2024-09-15T21:25:04.666064+0000","last_fullsized":"2024-09-15T21:25:04.666064+0000","mapping_epoch":51,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11863,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14266","reported_seq":14929,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:04.228584+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:04.228584+0000","last_peered":"2024-09-15T21:25:04.228584+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:04.228584+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16842752,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11913,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730193+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730193+0000","last_peered":"2024-09-15T21:25:03.730193+0000","last_clean":"2024-09-15T21:25:03.730193+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730193+0000","last_undegraded":"2024-09-15T21:25:03.730193+0000","last_fullsized":"2024-09-15T21:25:03.730193+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14448","reported_seq":21546,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:04.686221+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:04.686221+0000","last_peered":"2024-09-15T21:25:04.686221+0000","last_clean":"2024-09-15T21:25:04.686221+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:04.686221+0000","last_undegraded":"2024-09-15T21:25:04.686221+0000","last_fullsized":"2024-09-15T21:25:04.686221+0000","mapping_epoch":47,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080937,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273355393,"num_objects":66739,"num_object_clones":0,"num_object_copies":133478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24745,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201040,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15820,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712564736,"data_stored":712507886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160826,"ondisk_log_size":160826,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953567,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518168,"kb_used_data":243524,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93853672,"statfs":{"total":96636764160,"available":96106160128,"internally_reserved":0,"allocated":249368576,"data_stored":249067174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149794,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633476,"kb_used_data":254760,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93738364,"statfs":{"total":96636764160,"available":95988084736,"internally_reserved":0,"allocated":260874240,"data_stored":260582193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626512,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547388,"kb_used_data":200556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93824452,"statfs":{"total":96636764160,"available":96076238848,"internally_reserved":0,"allocated":205369344,"data_stored":205029673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259268608,"data_stored":259265129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":204771328,"data_stored":204725508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248524800,"data_stored":248517249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:12.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:12.830 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:12.901 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-15T21:25:12.480936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264290964,"num_objects":64528,"num_object_clones":0,"num_object_copies":129056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203255,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15966,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663508,"kb_used_data":679972,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281452012,"statfs":{"total":289910292480,"available":288206860288,"internally_reserved":0,"allocated":696291328,"data_stored":695352925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,4,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18620416,"num_objects":-4546,"num_object_clones":0,"num_object_copies":-9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.771989"},"pg_stats":[{"pgid":"2.f","version":"52'14418","reported_seq":15180,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.221841+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:09.221841+0000","last_peered":"2024-09-15T21:25:09.221841+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:09.221841+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769024,"num_objects":4094,"num_object_clones":0,"num_object_copies":8188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14418,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14600","reported_seq":14710,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.782462+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:09.782462+0000","last_peered":"2024-09-15T21:25:09.782462+0000","last_clean":"2024-09-15T21:25:09.782462+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:09.782462+0000","last_undegraded":"2024-09-15T21:25:09.782462+0000","last_fullsized":"2024-09-15T21:25:09.782462+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16547840,"num_objects":4040,"num_object_clones":0,"num_object_copies":8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13326,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14687","reported_seq":21616,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.727953+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:09.727953+0000","last_peered":"2024-09-15T21:25:09.727953+0000","last_clean":"2024-09-15T21:25:09.727953+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:09.727953+0000","last_undegraded":"2024-09-15T21:25:09.727953+0000","last_fullsized":"2024-09-15T21:25:09.727953+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129496,"num_objects":4183,"num_object_clones":0,"num_object_copies":8366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13410,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14571","reported_seq":14654,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.802910+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:09.802910+0000","last_peered":"2024-09-15T21:25:09.802910+0000","last_clean":"2024-09-15T21:25:09.802910+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:09.802910+0000","last_undegraded":"2024-09-15T21:25:09.802910+0000","last_fullsized":"2024-09-15T21:25:09.802910+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16379904,"num_objects":3999,"num_object_clones":0,"num_object_copies":7998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14571,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14645","reported_seq":14777,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.195428+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:09.195428+0000","last_peered":"2024-09-15T21:25:09.195428+0000","last_clean":"2024-09-15T21:25:09.195428+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:09.195428+0000","last_undegraded":"2024-09-15T21:25:09.195428+0000","last_fullsized":"2024-09-15T21:25:09.195428+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16703488,"num_objects":4078,"num_object_clones":0,"num_object_copies":8156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14420","reported_seq":15247,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.166236+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:09.166236+0000","last_peered":"2024-09-15T21:25:09.166236+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:09.166236+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16224256,"num_objects":3961,"num_object_clones":0,"num_object_copies":7922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11845,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14794","reported_seq":15545,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.245233+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:09.245233+0000","last_peered":"2024-09-15T21:25:09.245233+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:09.245233+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12430,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14374","reported_seq":14507,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.219609+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:09.219609+0000","last_peered":"2024-09-15T21:25:09.219609+0000","last_clean":"2024-09-15T21:25:09.219609+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:09.219609+0000","last_undegraded":"2024-09-15T21:25:09.219609+0000","last_fullsized":"2024-09-15T21:25:09.219609+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506880,"num_objects":4030,"num_object_clones":0,"num_object_copies":8060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12159,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14334","reported_seq":21161,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.726719+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:09.726719+0000","last_peered":"2024-09-15T21:25:09.726719+0000","last_clean":"2024-09-15T21:25:09.726719+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:09.726719+0000","last_undegraded":"2024-09-15T21:25:09.726719+0000","last_fullsized":"2024-09-15T21:25:09.726719+0000","mapping_epoch":49,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14186","reported_seq":14974,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.239387+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:09.239387+0000","last_peered":"2024-09-15T21:25:09.239387+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:09.239387+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15761408,"num_objects":3848,"num_object_clones":0,"num_object_copies":7696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14272","reported_seq":15107,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.829241+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:09.829241+0000","last_peered":"2024-09-15T21:25:09.829241+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:09.829241+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16437248,"num_objects":4013,"num_object_clones":0,"num_object_copies":8026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12823,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14461","reported_seq":14877,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.772177+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:09.772177+0000","last_peered":"2024-09-15T21:25:09.772177+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:09.772177+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740352,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14629","reported_seq":15377,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.196058+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:09.196058+0000","last_peered":"2024-09-15T21:25:09.196058+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:09.196058+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16715776,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11936,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14429","reported_seq":21347,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.607872+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:09.607872+0000","last_peered":"2024-09-15T21:25:09.607872+0000","last_clean":"2024-09-15T21:25:09.607872+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:09.607872+0000","last_undegraded":"2024-09-15T21:25:09.607872+0000","last_fullsized":"2024-09-15T21:25:09.607872+0000","mapping_epoch":51,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14415","reported_seq":15078,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.237034+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:09.237034+0000","last_peered":"2024-09-15T21:25:09.237034+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:09.237034+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232448,"num_objects":3963,"num_object_clones":0,"num_object_copies":7926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12062,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14604","reported_seq":21702,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.719859+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:09.719859+0000","last_peered":"2024-09-15T21:25:09.719859+0000","last_clean":"2024-09-15T21:25:09.719859+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:09.719859+0000","last_undegraded":"2024-09-15T21:25:09.719859+0000","last_fullsized":"2024-09-15T21:25:09.719859+0000","mapping_epoch":47,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441961,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12264,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264290945,"num_objects":64526,"num_object_clones":0,"num_object_copies":129052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203253,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15965,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694427648,"data_stored":694370798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501428,"kb_used_data":239268,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93870412,"statfs":{"total":96636764160,"available":96123301888,"internally_reserved":0,"allocated":245010432,"data_stored":244709030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149795,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633820,"kb_used_data":246796,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93738020,"statfs":{"total":96636764160,"available":95987732480,"internally_reserved":0,"allocated":252719104,"data_stored":252421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":396291761},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":193908,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":198561792,"data_stored":198222121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252280832,"data_stored":252277353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197898240,"data_stored":197852420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":244248576,"data_stored":244241025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:12.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:13.644 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:13.645 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:13.671 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-15T21:25:12.480936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264290964,"num_objects":64528,"num_object_clones":0,"num_object_copies":129056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203255,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15966,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663508,"kb_used_data":679972,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281452012,"statfs":{"total":289910292480,"available":288206860288,"internally_reserved":0,"allocated":696291328,"data_stored":695352925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,4,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18620416,"num_objects":-4546,"num_object_clones":0,"num_object_copies":-9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.771989"},"pg_stats":[{"pgid":"2.f","version":"52'14418","reported_seq":15180,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.221841+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:09.221841+0000","last_peered":"2024-09-15T21:25:09.221841+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:09.221841+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769024,"num_objects":4094,"num_object_clones":0,"num_object_copies":8188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14418,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14600","reported_seq":14710,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.782462+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:09.782462+0000","last_peered":"2024-09-15T21:25:09.782462+0000","last_clean":"2024-09-15T21:25:09.782462+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:09.782462+0000","last_undegraded":"2024-09-15T21:25:09.782462+0000","last_fullsized":"2024-09-15T21:25:09.782462+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16547840,"num_objects":4040,"num_object_clones":0,"num_object_copies":8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13326,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14687","reported_seq":21616,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.727953+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:09.727953+0000","last_peered":"2024-09-15T21:25:09.727953+0000","last_clean":"2024-09-15T21:25:09.727953+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:09.727953+0000","last_undegraded":"2024-09-15T21:25:09.727953+0000","last_fullsized":"2024-09-15T21:25:09.727953+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129496,"num_objects":4183,"num_object_clones":0,"num_object_copies":8366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13410,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14571","reported_seq":14654,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.802910+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:09.802910+0000","last_peered":"2024-09-15T21:25:09.802910+0000","last_clean":"2024-09-15T21:25:09.802910+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:09.802910+0000","last_undegraded":"2024-09-15T21:25:09.802910+0000","last_fullsized":"2024-09-15T21:25:09.802910+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16379904,"num_objects":3999,"num_object_clones":0,"num_object_copies":7998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14571,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14645","reported_seq":14777,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.195428+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:09.195428+0000","last_peered":"2024-09-15T21:25:09.195428+0000","last_clean":"2024-09-15T21:25:09.195428+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:09.195428+0000","last_undegraded":"2024-09-15T21:25:09.195428+0000","last_fullsized":"2024-09-15T21:25:09.195428+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16703488,"num_objects":4078,"num_object_clones":0,"num_object_copies":8156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14420","reported_seq":15247,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.166236+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:09.166236+0000","last_peered":"2024-09-15T21:25:09.166236+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:09.166236+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16224256,"num_objects":3961,"num_object_clones":0,"num_object_copies":7922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11845,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14794","reported_seq":15545,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.245233+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:09.245233+0000","last_peered":"2024-09-15T21:25:09.245233+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:09.245233+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12430,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14374","reported_seq":14507,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.219609+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:09.219609+0000","last_peered":"2024-09-15T21:25:09.219609+0000","last_clean":"2024-09-15T21:25:09.219609+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:09.219609+0000","last_undegraded":"2024-09-15T21:25:09.219609+0000","last_fullsized":"2024-09-15T21:25:09.219609+0000","mapping_epoch":35,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506880,"num_objects":4030,"num_object_clones":0,"num_object_copies":8060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12159,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14334","reported_seq":21161,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.726719+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:09.726719+0000","last_peered":"2024-09-15T21:25:09.726719+0000","last_clean":"2024-09-15T21:25:09.726719+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:09.726719+0000","last_undegraded":"2024-09-15T21:25:09.726719+0000","last_fullsized":"2024-09-15T21:25:09.726719+0000","mapping_epoch":49,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14186","reported_seq":14974,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.239387+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:09.239387+0000","last_peered":"2024-09-15T21:25:09.239387+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:09.239387+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4100","ondisk_log_start":"44'4100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15761408,"num_objects":3848,"num_object_clones":0,"num_object_copies":7696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14272","reported_seq":15107,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.829241+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:09.829241+0000","last_peered":"2024-09-15T21:25:09.829241+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:09.829241+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16437248,"num_objects":4013,"num_object_clones":0,"num_object_copies":8026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12823,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14461","reported_seq":14877,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.772177+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:09.772177+0000","last_peered":"2024-09-15T21:25:09.772177+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:09.772177+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740352,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14629","reported_seq":15377,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.196058+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:09.196058+0000","last_peered":"2024-09-15T21:25:09.196058+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:09.196058+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16715776,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11936,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14429","reported_seq":21347,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.607872+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:09.607872+0000","last_peered":"2024-09-15T21:25:09.607872+0000","last_clean":"2024-09-15T21:25:09.607872+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:09.607872+0000","last_undegraded":"2024-09-15T21:25:09.607872+0000","last_fullsized":"2024-09-15T21:25:09.607872+0000","mapping_epoch":51,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14415","reported_seq":15078,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.237034+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:09.237034+0000","last_peered":"2024-09-15T21:25:09.237034+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:09.237034+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16232448,"num_objects":3963,"num_object_clones":0,"num_object_copies":7926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12062,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14604","reported_seq":21702,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.719859+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:09.719859+0000","last_peered":"2024-09-15T21:25:09.719859+0000","last_clean":"2024-09-15T21:25:09.719859+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:09.719859+0000","last_undegraded":"2024-09-15T21:25:09.719859+0000","last_fullsized":"2024-09-15T21:25:09.719859+0000","mapping_epoch":47,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441961,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12264,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264290945,"num_objects":64526,"num_object_clones":0,"num_object_copies":129052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203253,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15965,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694427648,"data_stored":694370798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501428,"kb_used_data":239268,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93870412,"statfs":{"total":96636764160,"available":96123301888,"internally_reserved":0,"allocated":245010432,"data_stored":244709030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149795,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633820,"kb_used_data":246796,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93738020,"statfs":{"total":96636764160,"available":95987732480,"internally_reserved":0,"allocated":252719104,"data_stored":252421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":396291761},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":193908,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":198561792,"data_stored":198222121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252280832,"data_stored":252277353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197898240,"data_stored":197852420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":244248576,"data_stored":244241025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:13.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:13.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'14418", 'reported_seq': 15180, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.221841+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:09.221841+0000', 'last_peered': '2024-09-15T21:25:09.221841+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:09.221841+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16769024, 'num_objects': 4094, 'num_object_clones': 0, 'num_object_copies': 8188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4094, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14418, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1063}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:13.672 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'14420", 'reported_seq': 15247, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.166236+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:09.166236+0000', 'last_peered': '2024-09-15T21:25:09.166236+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:09.166236+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16224256, 'num_objects': 3961, 'num_object_clones': 0, 'num_object_copies': 7922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11845, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 932}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'14794", 'reported_seq': 15545, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.245233+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:09.245233+0000', 'last_peered': '2024-09-15T21:25:09.245233+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:09.245233+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16834560, 'num_objects': 4110, 'num_object_clones': 0, 'num_object_copies': 8220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12430, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 946}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14186", 'reported_seq': 14974, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.239387+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:09.239387+0000', 'last_peered': '2024-09-15T21:25:09.239387+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:09.239387+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4100", 'ondisk_log_start': "44'4100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15761408, 'num_objects': 3848, 'num_object_clones': 0, 'num_object_copies': 7696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11712, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 833}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'14272", 'reported_seq': 15107, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.829241+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:09.829241+0000', 'last_peered': '2024-09-15T21:25:09.829241+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:09.829241+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4200", 'ondisk_log_start': "44'4200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16437248, 'num_objects': 4013, 'num_object_clones': 0, 'num_object_copies': 8026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2912, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12823, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 356, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1101}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'14461", 'reported_seq': 14877, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.772177+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:09.772177+0000', 'last_peered': '2024-09-15T21:25:09.772177+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:09.772177+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16740352, 'num_objects': 4087, 'num_object_clones': 0, 'num_object_copies': 8174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12860, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 881}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:13.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'14629", 'reported_seq': 15377, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.196058+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:09.196058+0000', 'last_peered': '2024-09-15T21:25:09.196058+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:09.196058+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16715776, 'num_objects': 4081, 'num_object_clones': 0, 'num_object_copies': 8162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11936, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:13.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'14415", 'reported_seq': 15078, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:09.237034+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:09.237034+0000', 'last_peered': '2024-09-15T21:25:09.237034+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:09.237034+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16232448, 'num_objects': 3963, 'num_object_clones': 0, 'num_object_copies': 7926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12062, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 861}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:13.674 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:14.526 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:14.527 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24600/129056 objects degraded (19.061%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":64528,"data_bytes":264290964,"bytes_used":1703432192,"bytes_avail":288206860288,"bytes_total":289910292480,"degraded_objects":24600,"degraded_total":129056,"degraded_ratio":0.19061492685345896,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:14.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:14.556 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:15.130 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:15.132 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:15.211 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-15T21:25:14.481244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259707540,"num_objects":63409,"num_object_clones":0,"num_object_copies":126818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204374,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15966,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1644376,"kb_used_data":673312,"kb_used_omap":12,"kb_used_meta":970995,"kb_avail":281471144,"statfs":{"total":289910292480,"available":288226451456,"internally_reserved":0,"allocated":689471488,"data_stored":688533085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":994298890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,7,2,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23203840,"num_objects":-5665,"num_object_clones":0,"num_object_copies":-11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.760479"},"pg_stats":[{"pgid":"2.f","version":"52'14556","reported_seq":15318,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.309154+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:14.309154+0000","last_peered":"2024-09-15T21:25:14.309154+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:14.309154+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203776,"num_objects":3956,"num_object_clones":0,"num_object_copies":7912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14600","reported_seq":14710,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.782462+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:09.782462+0000","last_peered":"2024-09-15T21:25:09.782462+0000","last_clean":"2024-09-15T21:25:09.782462+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:09.782462+0000","last_undegraded":"2024-09-15T21:25:09.782462+0000","last_fullsized":"2024-09-15T21:25:09.782462+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16547840,"num_objects":4040,"num_object_clones":0,"num_object_copies":8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13326,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14687","reported_seq":21616,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.727953+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:09.727953+0000","last_peered":"2024-09-15T21:25:09.727953+0000","last_clean":"2024-09-15T21:25:09.727953+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:09.727953+0000","last_undegraded":"2024-09-15T21:25:09.727953+0000","last_fullsized":"2024-09-15T21:25:09.727953+0000","mapping_epoch":45,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129496,"num_objects":4183,"num_object_clones":0,"num_object_copies":8366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4183,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13410,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14571","reported_seq":14654,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.802910+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:09.802910+0000","last_peered":"2024-09-15T21:25:09.802910+0000","last_clean":"2024-09-15T21:25:09.802910+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:09.802910+0000","last_undegraded":"2024-09-15T21:25:09.802910+0000","last_fullsized":"2024-09-15T21:25:09.802910+0000","mapping_epoch":36,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16379904,"num_objects":3999,"num_object_clones":0,"num_object_copies":7998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14571,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14769","reported_seq":14901,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.287032+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:14.287032+0000","last_peered":"2024-09-15T21:25:14.287032+0000","last_clean":"2024-09-15T21:25:14.287032+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:14.287032+0000","last_undegraded":"2024-09-15T21:25:14.287032+0000","last_fullsized":"2024-09-15T21:25:14.287032+0000","mapping_epoch":35,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12627,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14561","reported_seq":15388,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.252748+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:14.252748+0000","last_peered":"2024-09-15T21:25:14.252748+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:14.252748+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15646720,"num_objects":3820,"num_object_clones":0,"num_object_copies":7640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11986,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14962","reported_seq":15713,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.305697+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:14.305697+0000","last_peered":"2024-09-15T21:25:14.305697+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:14.305697+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12598,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14507","reported_seq":14640,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.219782+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:14.219782+0000","last_peered":"2024-09-15T21:25:14.219782+0000","last_clean":"2024-09-15T21:25:14.219782+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:14.219782+0000","last_undegraded":"2024-09-15T21:25:14.219782+0000","last_fullsized":"2024-09-15T21:25:14.219782+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962112,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12292,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14334","reported_seq":21161,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.726719+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:09.726719+0000","last_peered":"2024-09-15T21:25:09.726719+0000","last_clean":"2024-09-15T21:25:09.726719+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:09.726719+0000","last_undegraded":"2024-09-15T21:25:09.726719+0000","last_fullsized":"2024-09-15T21:25:09.726719+0000","mapping_epoch":49,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14328","reported_seq":15116,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.280595+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:14.280595+0000","last_peered":"2024-09-15T21:25:14.280595+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:14.280595+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15179776,"num_objects":3706,"num_object_clones":0,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11854,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14272","reported_seq":15107,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.829241+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:09.829241+0000","last_peered":"2024-09-15T21:25:09.829241+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:09.829241+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4200","ondisk_log_start":"44'4200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16437248,"num_objects":4013,"num_object_clones":0,"num_object_copies":8026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12823,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1101}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14461","reported_seq":14877,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:09.772177+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:09.772177+0000","last_peered":"2024-09-15T21:25:09.772177+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:09.772177+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16740352,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12860,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14771","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.315654+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:14.315654+0000","last_peered":"2024-09-15T21:25:14.315654+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:14.315654+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12078,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14429","reported_seq":21347,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.607872+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:09.607872+0000","last_peered":"2024-09-15T21:25:09.607872+0000","last_clean":"2024-09-15T21:25:09.607872+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:09.607872+0000","last_undegraded":"2024-09-15T21:25:09.607872+0000","last_fullsized":"2024-09-15T21:25:09.607872+0000","mapping_epoch":51,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16314368,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14546","reported_seq":15209,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.267348+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:14.267348+0000","last_peered":"2024-09-15T21:25:14.267348+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:14.267348+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14604","reported_seq":21702,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:09.719859+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:09.719859+0000","last_peered":"2024-09-15T21:25:09.719859+0000","last_clean":"2024-09-15T21:25:09.719859+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:09.719859+0000","last_undegraded":"2024-09-15T21:25:09.719859+0000","last_fullsized":"2024-09-15T21:25:09.719859+0000","mapping_epoch":47,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441961,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12264,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259707521,"num_objects":63407,"num_object_clones":0,"num_object_copies":126814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204372,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15965,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":687546368,"data_stored":687489518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501428,"kb_used_data":239268,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93870412,"statfs":{"total":96636764160,"available":96123301888,"internally_reserved":0,"allocated":245010432,"data_stored":244709030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614688,"kb_used_data":240136,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93757152,"statfs":{"total":96636764160,"available":96007323648,"internally_reserved":0,"allocated":245899264,"data_stored":245601934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":193908,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":198561792,"data_stored":198222121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245399552,"data_stored":245396073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197898240,"data_stored":197852420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":244248576,"data_stored":244241025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:18.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:18.640 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:18.664 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-15T21:25:18.482138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":255165076,"num_objects":62300,"num_object_clones":0,"num_object_copies":124600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62300,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205483,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16082,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645556,"kb_used_data":662012,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281469964,"statfs":{"total":289910292480,"available":288225243136,"internally_reserved":0,"allocated":677900288,"data_stored":676961885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,8,3,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18190336,"num_objects":-4441,"num_object_clones":0,"num_object_copies":-8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4441,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002601"},"pg_stats":[{"pgid":"2.f","version":"52'14556","reported_seq":15318,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.309154+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:14.309154+0000","last_peered":"2024-09-15T21:25:14.309154+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:14.309154+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203776,"num_objects":3956,"num_object_clones":0,"num_object_copies":7912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14718","reported_seq":14828,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.768301+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:14.768301+0000","last_peered":"2024-09-15T21:25:14.768301+0000","last_clean":"2024-09-15T21:25:14.768301+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:14.768301+0000","last_undegraded":"2024-09-15T21:25:14.768301+0000","last_fullsized":"2024-09-15T21:25:14.768301+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13444,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14816","reported_seq":21745,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.792631+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:14.792631+0000","last_peered":"2024-09-15T21:25:14.792631+0000","last_clean":"2024-09-15T21:25:14.792631+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:14.792631+0000","last_undegraded":"2024-09-15T21:25:14.792631+0000","last_fullsized":"2024-09-15T21:25:14.792631+0000","mapping_epoch":45,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16601112,"num_objects":4054,"num_object_clones":0,"num_object_copies":8108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13539,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14714","reported_seq":14797,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.845324+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:14.845324+0000","last_peered":"2024-09-15T21:25:14.845324+0000","last_clean":"2024-09-15T21:25:14.845324+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:14.845324+0000","last_undegraded":"2024-09-15T21:25:14.845324+0000","last_fullsized":"2024-09-15T21:25:14.845324+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794176,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14769","reported_seq":14901,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.287032+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:14.287032+0000","last_peered":"2024-09-15T21:25:14.287032+0000","last_clean":"2024-09-15T21:25:14.287032+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:14.287032+0000","last_undegraded":"2024-09-15T21:25:14.287032+0000","last_fullsized":"2024-09-15T21:25:14.287032+0000","mapping_epoch":35,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12627,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14561","reported_seq":15388,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.252748+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:14.252748+0000","last_peered":"2024-09-15T21:25:14.252748+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:14.252748+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15646720,"num_objects":3820,"num_object_clones":0,"num_object_copies":7640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11986,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14962","reported_seq":15713,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.305697+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:14.305697+0000","last_peered":"2024-09-15T21:25:14.305697+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:14.305697+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12598,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14507","reported_seq":14640,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.219782+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:14.219782+0000","last_peered":"2024-09-15T21:25:14.219782+0000","last_clean":"2024-09-15T21:25:14.219782+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:14.219782+0000","last_undegraded":"2024-09-15T21:25:14.219782+0000","last_fullsized":"2024-09-15T21:25:14.219782+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962112,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12292,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14493","reported_seq":21320,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.400009+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:14.400009+0000","last_peered":"2024-09-15T21:25:14.400009+0000","last_clean":"2024-09-15T21:25:14.400009+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:14.400009+0000","last_undegraded":"2024-09-15T21:25:14.400009+0000","last_fullsized":"2024-09-15T21:25:14.400009+0000","mapping_epoch":49,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900672,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14328","reported_seq":15116,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.280595+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:14.280595+0000","last_peered":"2024-09-15T21:25:14.280595+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:14.280595+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15179776,"num_objects":3706,"num_object_clones":0,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11854,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14408","reported_seq":15476,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.857547+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:14.857547+0000","last_peered":"2024-09-15T21:25:14.857547+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:14.857547+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15880192,"num_objects":3877,"num_object_clones":0,"num_object_copies":7754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12959,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14588","reported_seq":15004,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.746987+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:14.746987+0000","last_peered":"2024-09-15T21:25:14.746987+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:14.746987+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12987,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14771","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.315654+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:14.315654+0000","last_peered":"2024-09-15T21:25:14.315654+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:14.315654+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12078,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14584","reported_seq":21502,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.783369+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:14.783369+0000","last_peered":"2024-09-15T21:25:14.783369+0000","last_clean":"2024-09-15T21:25:14.783369+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:14.783369+0000","last_undegraded":"2024-09-15T21:25:14.783369+0000","last_fullsized":"2024-09-15T21:25:14.783369+0000","mapping_epoch":51,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15679488,"num_objects":3828,"num_object_clones":0,"num_object_copies":7656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12169,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14546","reported_seq":15209,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.267348+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:14.267348+0000","last_peered":"2024-09-15T21:25:14.267348+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:14.267348+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14746","reported_seq":21844,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.557336+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:14.557336+0000","last_peered":"2024-09-15T21:25:14.557336+0000","last_clean":"2024-09-15T21:25:14.557336+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:14.557336+0000","last_undegraded":"2024-09-15T21:25:14.557336+0000","last_fullsized":"2024-09-15T21:25:14.557336+0000","mapping_epoch":47,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15860329,"num_objects":3873,"num_object_clones":0,"num_object_copies":7746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12406,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":255165057,"num_objects":62298,"num_object_clones":0,"num_object_copies":124596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62298,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205481,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16081,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676274176,"data_stored":676217326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501148,"kb_used_data":234828,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93870692,"statfs":{"total":96636764160,"available":96123588608,"internally_reserved":0,"allocated":240463872,"data_stored":240162470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614688,"kb_used_data":240136,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93757152,"statfs":{"total":96636764160,"available":96007323648,"internally_reserved":0,"allocated":245899264,"data_stored":245601934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529720,"kb_used_data":187048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93842120,"statfs":{"total":96636764160,"available":96094330880,"internally_reserved":0,"allocated":191537152,"data_stored":191197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245399552,"data_stored":245396073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191000576,"data_stored":190954756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239874048,"data_stored":239866497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:18.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:19.458 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:19.459 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:19.497 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-15T21:25:18.482138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":255165076,"num_objects":62300,"num_object_clones":0,"num_object_copies":124600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62300,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205483,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16082,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1645556,"kb_used_data":662012,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281469964,"statfs":{"total":289910292480,"available":288225243136,"internally_reserved":0,"allocated":677900288,"data_stored":676961885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1007078410},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,8,3,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18190336,"num_objects":-4441,"num_object_clones":0,"num_object_copies":-8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4441,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002601"},"pg_stats":[{"pgid":"2.f","version":"52'14556","reported_seq":15318,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.309154+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:14.309154+0000","last_peered":"2024-09-15T21:25:14.309154+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:14.309154+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16203776,"num_objects":3956,"num_object_clones":0,"num_object_copies":7912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14718","reported_seq":14828,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.768301+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:14.768301+0000","last_peered":"2024-09-15T21:25:14.768301+0000","last_clean":"2024-09-15T21:25:14.768301+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:14.768301+0000","last_undegraded":"2024-09-15T21:25:14.768301+0000","last_fullsized":"2024-09-15T21:25:14.768301+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13444,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14816","reported_seq":21745,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.792631+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:14.792631+0000","last_peered":"2024-09-15T21:25:14.792631+0000","last_clean":"2024-09-15T21:25:14.792631+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:14.792631+0000","last_undegraded":"2024-09-15T21:25:14.792631+0000","last_fullsized":"2024-09-15T21:25:14.792631+0000","mapping_epoch":45,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16601112,"num_objects":4054,"num_object_clones":0,"num_object_copies":8108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13539,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14714","reported_seq":14797,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.845324+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:14.845324+0000","last_peered":"2024-09-15T21:25:14.845324+0000","last_clean":"2024-09-15T21:25:14.845324+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:14.845324+0000","last_undegraded":"2024-09-15T21:25:14.845324+0000","last_fullsized":"2024-09-15T21:25:14.845324+0000","mapping_epoch":36,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15794176,"num_objects":3856,"num_object_clones":0,"num_object_copies":7712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14769","reported_seq":14901,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.287032+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:14.287032+0000","last_peered":"2024-09-15T21:25:14.287032+0000","last_clean":"2024-09-15T21:25:14.287032+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:14.287032+0000","last_undegraded":"2024-09-15T21:25:14.287032+0000","last_fullsized":"2024-09-15T21:25:14.287032+0000","mapping_epoch":35,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12627,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14561","reported_seq":15388,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.252748+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:14.252748+0000","last_peered":"2024-09-15T21:25:14.252748+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:14.252748+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15646720,"num_objects":3820,"num_object_clones":0,"num_object_copies":7640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11986,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'14962","reported_seq":15713,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.305697+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:14.305697+0000","last_peered":"2024-09-15T21:25:14.305697+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:14.305697+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12598,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14507","reported_seq":14640,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.219782+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:14.219782+0000","last_peered":"2024-09-15T21:25:14.219782+0000","last_clean":"2024-09-15T21:25:14.219782+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:14.219782+0000","last_undegraded":"2024-09-15T21:25:14.219782+0000","last_fullsized":"2024-09-15T21:25:14.219782+0000","mapping_epoch":35,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962112,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12292,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14493","reported_seq":21320,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.400009+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:14.400009+0000","last_peered":"2024-09-15T21:25:14.400009+0000","last_clean":"2024-09-15T21:25:14.400009+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:14.400009+0000","last_undegraded":"2024-09-15T21:25:14.400009+0000","last_fullsized":"2024-09-15T21:25:14.400009+0000","mapping_epoch":49,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900672,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13079,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14328","reported_seq":15116,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.280595+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:14.280595+0000","last_peered":"2024-09-15T21:25:14.280595+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:14.280595+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4300","ondisk_log_start":"44'4300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15179776,"num_objects":3706,"num_object_clones":0,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11854,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":691}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14408","reported_seq":15476,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.857547+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:14.857547+0000","last_peered":"2024-09-15T21:25:14.857547+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:14.857547+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15880192,"num_objects":3877,"num_object_clones":0,"num_object_copies":7754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12959,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14588","reported_seq":15004,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.746987+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:14.746987+0000","last_peered":"2024-09-15T21:25:14.746987+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:14.746987+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16220160,"num_objects":3960,"num_object_clones":0,"num_object_copies":7920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12987,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14771","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.315654+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:14.315654+0000","last_peered":"2024-09-15T21:25:14.315654+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:14.315654+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12078,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14584","reported_seq":21502,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.783369+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:14.783369+0000","last_peered":"2024-09-15T21:25:14.783369+0000","last_clean":"2024-09-15T21:25:14.783369+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:14.783369+0000","last_undegraded":"2024-09-15T21:25:14.783369+0000","last_fullsized":"2024-09-15T21:25:14.783369+0000","mapping_epoch":51,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15679488,"num_objects":3828,"num_object_clones":0,"num_object_copies":7656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12169,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14546","reported_seq":15209,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:14.267348+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:14.267348+0000","last_peered":"2024-09-15T21:25:14.267348+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:14.267348+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15695872,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12193,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14746","reported_seq":21844,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:14.557336+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:14.557336+0000","last_peered":"2024-09-15T21:25:14.557336+0000","last_clean":"2024-09-15T21:25:14.557336+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:14.557336+0000","last_undegraded":"2024-09-15T21:25:14.557336+0000","last_fullsized":"2024-09-15T21:25:14.557336+0000","mapping_epoch":47,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15860329,"num_objects":3873,"num_object_clones":0,"num_object_copies":7746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12406,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":255165057,"num_objects":62298,"num_object_clones":0,"num_object_copies":124596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62298,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205481,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16081,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676274176,"data_stored":676217326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953569,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501148,"kb_used_data":234828,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93870692,"statfs":{"total":96636764160,"available":96123588608,"internally_reserved":0,"allocated":240463872,"data_stored":240162470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149796,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614688,"kb_used_data":240136,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93757152,"statfs":{"total":96636764160,"available":96007323648,"internally_reserved":0,"allocated":245899264,"data_stored":245601934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":383512241},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529720,"kb_used_data":187048,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93842120,"statfs":{"total":96636764160,"available":96094330880,"internally_reserved":0,"allocated":191537152,"data_stored":191197481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":350875308},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245399552,"data_stored":245396073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":191000576,"data_stored":190954756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239874048,"data_stored":239866497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'14556", 'reported_seq': 15318, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.309154+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:14.309154+0000', 'last_peered': '2024-09-15T21:25:14.309154+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:14.309154+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16203776, 'num_objects': 3956, 'num_object_clones': 0, 'num_object_copies': 7912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14556, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 925}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'14561", 'reported_seq': 15388, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.252748+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:14.252748+0000', 'last_peered': '2024-09-15T21:25:14.252748+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:14.252748+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15646720, 'num_objects': 3820, 'num_object_clones': 0, 'num_object_copies': 7640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11986, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:19.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'14962", 'reported_seq': 15713, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.305697+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:14.305697+0000', 'last_peered': '2024-09-15T21:25:14.305697+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:14.305697+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16146432, 'num_objects': 3942, 'num_object_clones': 0, 'num_object_copies': 7884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12598, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14328", 'reported_seq': 15116, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.280595+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:14.280595+0000', 'last_peered': '2024-09-15T21:25:14.280595+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:14.280595+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4300", 'ondisk_log_start': "44'4300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15179776, 'num_objects': 3706, 'num_object_clones': 0, 'num_object_copies': 7412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11854, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 691}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'14408", 'reported_seq': 15476, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.857547+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:14.857547+0000', 'last_peered': '2024-09-15T21:25:14.857547+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:14.857547+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4400", 'ondisk_log_start': "44'4400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15880192, 'num_objects': 3877, 'num_object_clones': 0, 'num_object_copies': 7754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2796, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12959, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 472, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1081}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:19.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'14588", 'reported_seq': 15004, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.746987+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:14.746987+0000', 'last_peered': '2024-09-15T21:25:14.746987+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:14.746987+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16220160, 'num_objects': 3960, 'num_object_clones': 0, 'num_object_copies': 7920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12987, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 754}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:19.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:19.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'14771", 'reported_seq': 15519, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.315654+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:14.315654+0000', 'last_peered': '2024-09-15T21:25:14.315654+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:14.315654+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16134144, 'num_objects': 3939, 'num_object_clones': 0, 'num_object_copies': 7878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3939, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12078, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:19.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'14546", 'reported_seq': 15209, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:14.267348+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:14.267348+0000', 'last_peered': '2024-09-15T21:25:14.267348+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:14.267348+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15695872, 'num_objects': 3832, 'num_object_clones': 0, 'num_object_copies': 7664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12193, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:20.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:20.228 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24484/124600 objects degraded (19.650%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":533,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62300,"data_bytes":255165076,"bytes_used":1685049344,"bytes_avail":288225243136,"bytes_total":289910292480,"degraded_objects":24484,"degraded_total":124600,"degraded_ratio":0.19650080256821831,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-15T21:24:40.467113+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:20.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:20.272 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:21.023 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:21.070 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-15T21:25:20.483485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245678740,"num_objects":59984,"num_object_clones":0,"num_object_copies":119968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207799,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16198,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160586,"ondisk_log_size":160586,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1639584,"kb_used_data":643568,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281475936,"statfs":{"total":289910292480,"available":288231358464,"internally_reserved":0,"allocated":659013632,"data_stored":658075229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1019857930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,5,9,5,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27676672,"num_objects":-6757,"num_object_clones":0,"num_object_copies":-13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003469"},"pg_stats":[{"pgid":"2.f","version":"52'14707","reported_seq":15469,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.291188+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:19.291188+0000","last_peered":"2024-09-15T21:25:19.291188+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:19.291188+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15585280,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14707,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14867","reported_seq":14977,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.746259+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:19.746259+0000","last_peered":"2024-09-15T21:25:19.746259+0000","last_clean":"2024-09-15T21:25:19.746259+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:19.746259+0000","last_undegraded":"2024-09-15T21:25:19.746259+0000","last_fullsized":"2024-09-15T21:25:19.746259+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15454208,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14961","reported_seq":21890,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.570461+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:19.570461+0000","last_peered":"2024-09-15T21:25:19.570461+0000","last_clean":"2024-09-15T21:25:19.570461+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:19.570461+0000","last_undegraded":"2024-09-15T21:25:19.570461+0000","last_fullsized":"2024-09-15T21:25:19.570461+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007192,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13684,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14850","reported_seq":14933,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.740753+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:19.740753+0000","last_peered":"2024-09-15T21:25:19.740753+0000","last_clean":"2024-09-15T21:25:19.740753+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:19.740753+0000","last_undegraded":"2024-09-15T21:25:19.740753+0000","last_fullsized":"2024-09-15T21:25:19.740753+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'14923","reported_seq":15055,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.314268+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:19.314268+0000","last_peered":"2024-09-15T21:25:19.314268+0000","last_clean":"2024-09-15T21:25:19.314268+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:19.314268+0000","last_undegraded":"2024-09-15T21:25:19.314268+0000","last_fullsized":"2024-09-15T21:25:19.314268+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15564800,"num_objects":3800,"num_object_clones":0,"num_object_copies":7600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12781,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14686","reported_seq":15510,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.242092+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:19.242092+0000","last_peered":"2024-09-15T21:25:19.242092+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:19.242092+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15134720,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12111,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":666}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15101","reported_seq":15852,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.257132+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:19.257132+0000","last_peered":"2024-09-15T21:25:19.257132+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:19.257132+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15577088,"num_objects":3803,"num_object_clones":0,"num_object_copies":7606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12737,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14643","reported_seq":14776,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.315162+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:19.315162+0000","last_peered":"2024-09-15T21:25:19.315162+0000","last_clean":"2024-09-15T21:25:19.315162+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:19.315162+0000","last_undegraded":"2024-09-15T21:25:19.315162+0000","last_fullsized":"2024-09-15T21:25:19.315162+0000","mapping_epoch":35,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405056,"num_objects":3761,"num_object_clones":0,"num_object_copies":7522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14642","reported_seq":21469,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.571359+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:19.571359+0000","last_peered":"2024-09-15T21:25:19.571359+0000","last_clean":"2024-09-15T21:25:19.571359+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:19.571359+0000","last_undegraded":"2024-09-15T21:25:19.571359+0000","last_fullsized":"2024-09-15T21:25:19.571359+0000","mapping_epoch":49,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14463","reported_seq":15251,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.297932+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:19.297932+0000","last_peered":"2024-09-15T21:25:19.297932+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:19.297932+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4400","ondisk_log_start":"44'4400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14626816,"num_objects":3571,"num_object_clones":0,"num_object_copies":7142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11989,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14545","reported_seq":15845,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.834507+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:19.834507+0000","last_peered":"2024-09-15T21:25:19.834507+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:19.834507+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15319040,"num_objects":3740,"num_object_clones":0,"num_object_copies":7480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14737","reported_seq":15153,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.706061+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:19.706061+0000","last_peered":"2024-09-15T21:25:19.706061+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:19.706061+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13136,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'14933","reported_seq":15681,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.316915+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:19.316915+0000","last_peered":"2024-09-15T21:25:19.316915+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:19.316915+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14719","reported_seq":21637,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.440430+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:19.440430+0000","last_peered":"2024-09-15T21:25:19.440430+0000","last_clean":"2024-09-15T21:25:19.440430+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:19.440430+0000","last_undegraded":"2024-09-15T21:25:19.440430+0000","last_fullsized":"2024-09-15T21:25:19.440430+0000","mapping_epoch":51,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14705","reported_seq":15368,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.338136+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:19.338136+0000","last_peered":"2024-09-15T21:25:19.338136+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:19.338136+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15044608,"num_objects":3673,"num_object_clones":0,"num_object_copies":7346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12352,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14901","reported_seq":21999,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.572736+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:19.572736+0000","last_peered":"2024-09-15T21:25:19.572736+0000","last_clean":"2024-09-15T21:25:19.572736+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:19.572736+0000","last_undegraded":"2024-09-15T21:25:19.572736+0000","last_fullsized":"2024-09-15T21:25:19.572736+0000","mapping_epoch":47,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15225449,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245678721,"num_objects":59982,"num_object_clones":0,"num_object_copies":119964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207797,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16197,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":657121280,"data_stored":657064430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160583,"ondisk_log_size":160583,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953570,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500696,"kb_used_data":230216,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93871144,"statfs":{"total":96636764160,"available":96124051456,"internally_reserved":0,"allocated":235741184,"data_stored":235439782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149797,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611936,"kb_used_data":233232,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93759904,"statfs":{"total":96636764160,"available":96010141696,"internally_reserved":0,"allocated":238829568,"data_stored":238532238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":387772081},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,2,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526952,"kb_used_data":180120,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93844888,"statfs":{"total":96636764160,"available":96097165312,"internally_reserved":0,"allocated":184442880,"data_stored":184103209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":238223360,"data_stored":238219881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183783424,"data_stored":183737604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235114496,"data_stored":235106945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:24.072 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:24.931 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:25.003 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-15T21:25:24.485696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240747156,"num_objects":58780,"num_object_clones":0,"num_object_copies":117560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209003,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16198,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160690,"ondisk_log_size":160690,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642176,"kb_used_data":636432,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281473344,"statfs":{"total":289910292480,"available":288228704256,"internally_reserved":0,"allocated":651706368,"data_stored":650767965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1029819402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,6,12,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23543808,"num_objects":-5748,"num_object_clones":0,"num_object_copies":-11496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004759"},"pg_stats":[{"pgid":"2.f","version":"52'14845","reported_seq":15607,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.261780+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:24.261780+0000","last_peered":"2024-09-15T21:25:24.261780+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:24.261780+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15020032,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14867","reported_seq":14977,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.746259+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:19.746259+0000","last_peered":"2024-09-15T21:25:19.746259+0000","last_clean":"2024-09-15T21:25:19.746259+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:19.746259+0000","last_undegraded":"2024-09-15T21:25:19.746259+0000","last_fullsized":"2024-09-15T21:25:19.746259+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15454208,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14961","reported_seq":21890,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.570461+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:19.570461+0000","last_peered":"2024-09-15T21:25:19.570461+0000","last_clean":"2024-09-15T21:25:19.570461+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:19.570461+0000","last_undegraded":"2024-09-15T21:25:19.570461+0000","last_fullsized":"2024-09-15T21:25:19.570461+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007192,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13684,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14850","reported_seq":14933,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.740753+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:19.740753+0000","last_peered":"2024-09-15T21:25:19.740753+0000","last_clean":"2024-09-15T21:25:19.740753+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:19.740753+0000","last_undegraded":"2024-09-15T21:25:19.740753+0000","last_fullsized":"2024-09-15T21:25:19.740753+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15067","reported_seq":15199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.314191+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:24.314191+0000","last_peered":"2024-09-15T21:25:24.314191+0000","last_clean":"2024-09-15T21:25:24.314191+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:24.314191+0000","last_undegraded":"2024-09-15T21:25:24.314191+0000","last_fullsized":"2024-09-15T21:25:24.314191+0000","mapping_epoch":35,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14974976,"num_objects":3656,"num_object_clones":0,"num_object_copies":7312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14851","reported_seq":15678,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.310488+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:24.310488+0000","last_peered":"2024-09-15T21:25:24.310488+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:24.310488+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14458880,"num_objects":3530,"num_object_clones":0,"num_object_copies":7060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12276,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15262","reported_seq":16013,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.288964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:24.288964+0000","last_peered":"2024-09-15T21:25:24.288964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:24.288964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12898,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14816","reported_seq":14949,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.358696+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:24.358696+0000","last_peered":"2024-09-15T21:25:24.358696+0000","last_clean":"2024-09-15T21:25:24.358696+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:24.358696+0000","last_undegraded":"2024-09-15T21:25:24.358696+0000","last_fullsized":"2024-09-15T21:25:24.358696+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14642","reported_seq":21469,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.571359+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:19.571359+0000","last_peered":"2024-09-15T21:25:19.571359+0000","last_clean":"2024-09-15T21:25:19.571359+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:19.571359+0000","last_undegraded":"2024-09-15T21:25:19.571359+0000","last_fullsized":"2024-09-15T21:25:19.571359+0000","mapping_epoch":49,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14601","reported_seq":15389,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.344506+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:24.344506+0000","last_peered":"2024-09-15T21:25:24.344506+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:24.344506+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14061568,"num_objects":3433,"num_object_clones":0,"num_object_copies":6866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12127,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14545","reported_seq":15845,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.834507+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:19.834507+0000","last_peered":"2024-09-15T21:25:19.834507+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:19.834507+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15319040,"num_objects":3740,"num_object_clones":0,"num_object_copies":7480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14737","reported_seq":15153,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.706061+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:19.706061+0000","last_peered":"2024-09-15T21:25:19.706061+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:19.706061+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13136,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15067","reported_seq":15815,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.316553+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:24.316553+0000","last_peered":"2024-09-15T21:25:24.316553+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:24.316553+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14921728,"num_objects":3643,"num_object_clones":0,"num_object_copies":7286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12374,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14719","reported_seq":21637,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.440430+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:19.440430+0000","last_peered":"2024-09-15T21:25:19.440430+0000","last_clean":"2024-09-15T21:25:19.440430+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:19.440430+0000","last_undegraded":"2024-09-15T21:25:19.440430+0000","last_fullsized":"2024-09-15T21:25:19.440430+0000","mapping_epoch":51,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14856","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.184687+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:24.184687+0000","last_peered":"2024-09-15T21:25:24.184687+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:24.184687+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14426112,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14901","reported_seq":21999,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.572736+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:19.572736+0000","last_peered":"2024-09-15T21:25:19.572736+0000","last_clean":"2024-09-15T21:25:19.572736+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:19.572736+0000","last_undegraded":"2024-09-15T21:25:19.572736+0000","last_fullsized":"2024-09-15T21:25:19.572736+0000","mapping_epoch":47,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15225449,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240747137,"num_objects":58778,"num_object_clones":0,"num_object_copies":117556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209001,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16197,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649801728,"data_stored":649744878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160687,"ondisk_log_size":160687,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953570,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500696,"kb_used_data":230216,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93871144,"statfs":{"total":96636764160,"available":96124051456,"internally_reserved":0,"allocated":235741184,"data_stored":235439782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614528,"kb_used_data":226096,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93757312,"statfs":{"total":96636764160,"available":96007487488,"internally_reserved":0,"allocated":231522304,"data_stored":231224974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":397733553},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526952,"kb_used_data":180120,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93844888,"statfs":{"total":96636764160,"available":96097165312,"internally_reserved":0,"allocated":184442880,"data_stored":184103209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230903808,"data_stored":230900329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183783424,"data_stored":183737604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235114496,"data_stored":235106945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:25.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:25.847 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:25.849 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:25.884 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-15T21:25:24.485696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240747156,"num_objects":58780,"num_object_clones":0,"num_object_copies":117560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209003,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16198,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160690,"ondisk_log_size":160690,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642176,"kb_used_data":636432,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281473344,"statfs":{"total":289910292480,"available":288228704256,"internally_reserved":0,"allocated":651706368,"data_stored":650767965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1029819402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,6,12,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23543808,"num_objects":-5748,"num_object_clones":0,"num_object_copies":-11496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004759"},"pg_stats":[{"pgid":"2.f","version":"52'14845","reported_seq":15607,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.261780+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:24.261780+0000","last_peered":"2024-09-15T21:25:24.261780+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:24.261780+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15020032,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'14867","reported_seq":14977,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.746259+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:19.746259+0000","last_peered":"2024-09-15T21:25:19.746259+0000","last_clean":"2024-09-15T21:25:19.746259+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:19.746259+0000","last_undegraded":"2024-09-15T21:25:19.746259+0000","last_fullsized":"2024-09-15T21:25:19.746259+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15454208,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'14961","reported_seq":21890,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.570461+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:19.570461+0000","last_peered":"2024-09-15T21:25:19.570461+0000","last_clean":"2024-09-15T21:25:19.570461+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:19.570461+0000","last_undegraded":"2024-09-15T21:25:19.570461+0000","last_fullsized":"2024-09-15T21:25:19.570461+0000","mapping_epoch":45,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007192,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13684,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'14850","reported_seq":14933,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.740753+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:19.740753+0000","last_peered":"2024-09-15T21:25:19.740753+0000","last_clean":"2024-09-15T21:25:19.740753+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:19.740753+0000","last_undegraded":"2024-09-15T21:25:19.740753+0000","last_fullsized":"2024-09-15T21:25:19.740753+0000","mapping_epoch":36,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15237120,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15067","reported_seq":15199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.314191+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:24.314191+0000","last_peered":"2024-09-15T21:25:24.314191+0000","last_clean":"2024-09-15T21:25:24.314191+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:24.314191+0000","last_undegraded":"2024-09-15T21:25:24.314191+0000","last_fullsized":"2024-09-15T21:25:24.314191+0000","mapping_epoch":35,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14974976,"num_objects":3656,"num_object_clones":0,"num_object_copies":7312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14851","reported_seq":15678,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.310488+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:24.310488+0000","last_peered":"2024-09-15T21:25:24.310488+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:24.310488+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14458880,"num_objects":3530,"num_object_clones":0,"num_object_copies":7060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12276,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15262","reported_seq":16013,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.288964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:24.288964+0000","last_peered":"2024-09-15T21:25:24.288964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:24.288964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12898,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14816","reported_seq":14949,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.358696+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:24.358696+0000","last_peered":"2024-09-15T21:25:24.358696+0000","last_clean":"2024-09-15T21:25:24.358696+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:24.358696+0000","last_undegraded":"2024-09-15T21:25:24.358696+0000","last_fullsized":"2024-09-15T21:25:24.358696+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14642","reported_seq":21469,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.571359+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:19.571359+0000","last_peered":"2024-09-15T21:25:19.571359+0000","last_clean":"2024-09-15T21:25:19.571359+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:19.571359+0000","last_undegraded":"2024-09-15T21:25:19.571359+0000","last_fullsized":"2024-09-15T21:25:19.571359+0000","mapping_epoch":49,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15290368,"num_objects":3733,"num_object_clones":0,"num_object_copies":7466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13228,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14601","reported_seq":15389,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.344506+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:24.344506+0000","last_peered":"2024-09-15T21:25:24.344506+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:24.344506+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14061568,"num_objects":3433,"num_object_clones":0,"num_object_copies":6866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12127,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14545","reported_seq":15845,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.834507+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:19.834507+0000","last_peered":"2024-09-15T21:25:19.834507+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:19.834507+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4500","ondisk_log_start":"44'4500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15319040,"num_objects":3740,"num_object_clones":0,"num_object_copies":7480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14737","reported_seq":15153,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:19.706061+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:19.706061+0000","last_peered":"2024-09-15T21:25:19.706061+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:19.706061+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13136,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15067","reported_seq":15815,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.316553+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:24.316553+0000","last_peered":"2024-09-15T21:25:24.316553+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:24.316553+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14921728,"num_objects":3643,"num_object_clones":0,"num_object_copies":7286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12374,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14719","reported_seq":21637,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.440430+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:19.440430+0000","last_peered":"2024-09-15T21:25:19.440430+0000","last_clean":"2024-09-15T21:25:19.440430+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:19.440430+0000","last_undegraded":"2024-09-15T21:25:19.440430+0000","last_fullsized":"2024-09-15T21:25:19.440430+0000","mapping_epoch":51,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15126528,"num_objects":3693,"num_object_clones":0,"num_object_copies":7386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12304,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14856","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.184687+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:24.184687+0000","last_peered":"2024-09-15T21:25:24.184687+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:24.184687+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14426112,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'14901","reported_seq":21999,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:19.572736+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:19.572736+0000","last_peered":"2024-09-15T21:25:19.572736+0000","last_clean":"2024-09-15T21:25:19.572736+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:19.572736+0000","last_undegraded":"2024-09-15T21:25:19.572736+0000","last_fullsized":"2024-09-15T21:25:19.572736+0000","mapping_epoch":47,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15225449,"num_objects":3718,"num_object_clones":0,"num_object_copies":7436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240747137,"num_objects":58778,"num_object_clones":0,"num_object_copies":117556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209001,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16197,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649801728,"data_stored":649744878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160687,"ondisk_log_size":160687,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953570,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500696,"kb_used_data":230216,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93871144,"statfs":{"total":96636764160,"available":96124051456,"internally_reserved":0,"allocated":235741184,"data_stored":235439782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614528,"kb_used_data":226096,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93757312,"statfs":{"total":96636764160,"available":96007487488,"internally_reserved":0,"allocated":231522304,"data_stored":231224974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":397733553},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526952,"kb_used_data":180120,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93844888,"statfs":{"total":96636764160,"available":96097165312,"internally_reserved":0,"allocated":184442880,"data_stored":184103209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230903808,"data_stored":230900329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":183783424,"data_stored":183737604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235114496,"data_stored":235106945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:25.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:25.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'14845", 'reported_seq': 15607, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.261780+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:24.261780+0000', 'last_peered': '2024-09-15T21:25:24.261780+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:24.261780+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15020032, 'num_objects': 3667, 'num_object_clones': 0, 'num_object_copies': 7334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14845, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 636}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:25.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'14851", 'reported_seq': 15678, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.310488+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:24.310488+0000', 'last_peered': '2024-09-15T21:25:24.310488+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:24.310488+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14458880, 'num_objects': 3530, 'num_object_clones': 0, 'num_object_copies': 7060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12276, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 501}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'15262", 'reported_seq': 16013, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.288964+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:24.288964+0000', 'last_peered': '2024-09-15T21:25:24.288964+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:24.288964+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14917632, 'num_objects': 3642, 'num_object_clones': 0, 'num_object_copies': 7284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12898, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14601", 'reported_seq': 15389, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.344506+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:24.344506+0000', 'last_peered': '2024-09-15T21:25:24.344506+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:24.344506+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4600", 'ondisk_log_start': "44'4600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14061568, 'num_objects': 3433, 'num_object_clones': 0, 'num_object_copies': 6866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12127, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 418}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'14545", 'reported_seq': 15845, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:19.834507+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:19.834507+0000', 'last_peered': '2024-09-15T21:25:19.834507+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:19.834507+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4500", 'ondisk_log_start': "44'4500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15319040, 'num_objects': 3740, 'num_object_clones': 0, 'num_object_copies': 7480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2680, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13096, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 588, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1060}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'14737", 'reported_seq': 15153, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:19.706061+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:19.706061+0000', 'last_peered': '2024-09-15T21:25:19.706061+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:19.706061+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15609856, 'num_objects': 3811, 'num_object_clones': 0, 'num_object_copies': 7622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13136, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15067", 'reported_seq': 15815, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.316553+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:24.316553+0000', 'last_peered': '2024-09-15T21:25:24.316553+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:24.316553+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14921728, 'num_objects': 3643, 'num_object_clones': 0, 'num_object_copies': 7286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12374, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:25.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'14856", 'reported_seq': 15519, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:24.184687+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:24.184687+0000', 'last_peered': '2024-09-15T21:25:24.184687+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:24.184687+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14426112, 'num_objects': 3522, 'num_object_clones': 0, 'num_object_copies': 7044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12503, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.887 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:26.854 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:26.854 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24282/114980 objects degraded (21.118%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":539,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57490,"data_bytes":235463316,"bytes_used":1664774144,"bytes_avail":288245518336,"bytes_total":289910292480,"degraded_objects":24282,"degraded_total":114980,"degraded_ratio":0.21118455383544965,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":493},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:26.937 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:26.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:27.574 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:27.576 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:27.630 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-15T21:25:26.486110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235463316,"num_objects":57490,"num_object_clones":0,"num_object_copies":114980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210293,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16284,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1625756,"kb_used_data":624172,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281489764,"statfs":{"total":289910292480,"available":288245518336,"internally_reserved":0,"allocated":639152128,"data_stored":638213725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1025559562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,9,6,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24244224,"num_objects":-5919,"num_object_clones":0,"num_object_copies":-11838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-318,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5919,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004865"},"pg_stats":[{"pgid":"2.f","version":"52'14845","reported_seq":15607,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.261780+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:24.261780+0000","last_peered":"2024-09-15T21:25:24.261780+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:24.261780+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15020032,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":636}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15025","reported_seq":15135,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.818807+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:24.818807+0000","last_peered":"2024-09-15T21:25:24.818807+0000","last_clean":"2024-09-15T21:25:24.818807+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:24.818807+0000","last_undegraded":"2024-09-15T21:25:24.818807+0000","last_fullsized":"2024-09-15T21:25:24.818807+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14807040,"num_objects":3615,"num_object_clones":0,"num_object_copies":7230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13751,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15126","reported_seq":22055,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.853333+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:24.853333+0000","last_peered":"2024-09-15T21:25:24.853333+0000","last_clean":"2024-09-15T21:25:24.853333+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:24.853333+0000","last_undegraded":"2024-09-15T21:25:24.853333+0000","last_fullsized":"2024-09-15T21:25:24.853333+0000","mapping_epoch":45,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15331352,"num_objects":3744,"num_object_clones":0,"num_object_copies":7488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13849,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15012","reported_seq":15093,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.865494+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:24.865494+0000","last_peered":"2024-09-15T21:25:24.865494+0000","last_clean":"2024-09-15T21:25:24.865494+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:24.865494+0000","last_undegraded":"2024-09-15T21:25:24.865494+0000","last_fullsized":"2024-09-15T21:25:24.865494+0000","mapping_epoch":36,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14573568,"num_objects":3558,"num_object_clones":0,"num_object_copies":7116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15012,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15067","reported_seq":15199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.314191+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:24.314191+0000","last_peered":"2024-09-15T21:25:24.314191+0000","last_clean":"2024-09-15T21:25:24.314191+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:24.314191+0000","last_undegraded":"2024-09-15T21:25:24.314191+0000","last_fullsized":"2024-09-15T21:25:24.314191+0000","mapping_epoch":35,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14974976,"num_objects":3656,"num_object_clones":0,"num_object_copies":7312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'14851","reported_seq":15678,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.310488+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:24.310488+0000","last_peered":"2024-09-15T21:25:24.310488+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:24.310488+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14458880,"num_objects":3530,"num_object_clones":0,"num_object_copies":7060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12276,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15262","reported_seq":16013,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.288964+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:24.288964+0000","last_peered":"2024-09-15T21:25:24.288964+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:24.288964+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14917632,"num_objects":3642,"num_object_clones":0,"num_object_copies":7284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12898,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14816","reported_seq":14949,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.358696+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:24.358696+0000","last_peered":"2024-09-15T21:25:24.358696+0000","last_clean":"2024-09-15T21:25:24.358696+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:24.358696+0000","last_undegraded":"2024-09-15T21:25:24.358696+0000","last_fullsized":"2024-09-15T21:25:24.358696+0000","mapping_epoch":35,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12601,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14788","reported_seq":21615,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.820499+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:24.820499+0000","last_peered":"2024-09-15T21:25:24.820499+0000","last_clean":"2024-09-15T21:25:24.820499+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:24.820499+0000","last_undegraded":"2024-09-15T21:25:24.820499+0000","last_fullsized":"2024-09-15T21:25:24.820499+0000","mapping_epoch":49,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14692352,"num_objects":3587,"num_object_clones":0,"num_object_copies":7174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13374,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14601","reported_seq":15389,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.344506+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:24.344506+0000","last_peered":"2024-09-15T21:25:24.344506+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:24.344506+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4600","ondisk_log_start":"44'4600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14061568,"num_objects":3433,"num_object_clones":0,"num_object_copies":6866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12127,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14721","reported_seq":16193,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.888857+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:24.888857+0000","last_peered":"2024-09-15T21:25:24.888857+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:24.888857+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14598144,"num_objects":3564,"num_object_clones":0,"num_object_copies":7128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13272,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":970}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'14922","reported_seq":15338,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.884478+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:24.884478+0000","last_peered":"2024-09-15T21:25:24.884478+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:24.884478+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13321,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15067","reported_seq":15815,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.316553+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:24.316553+0000","last_peered":"2024-09-15T21:25:24.316553+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:24.316553+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14921728,"num_objects":3643,"num_object_clones":0,"num_object_copies":7286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12374,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'14872","reported_seq":21790,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.786696+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:24.786696+0000","last_peered":"2024-09-15T21:25:24.786696+0000","last_clean":"2024-09-15T21:25:24.786696+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:24.786696+0000","last_undegraded":"2024-09-15T21:25:24.786696+0000","last_fullsized":"2024-09-15T21:25:24.786696+0000","mapping_epoch":51,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14499840,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12457,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14856","reported_seq":15519,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:24.184687+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:24.184687+0000","last_peered":"2024-09-15T21:25:24.184687+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:24.184687+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14426112,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":420}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15046","reported_seq":22144,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:24.813616+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:24.813616+0000","last_peered":"2024-09-15T21:25:24.813616+0000","last_clean":"2024-09-15T21:25:24.813616+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:24.813616+0000","last_undegraded":"2024-09-15T21:25:24.813616+0000","last_fullsized":"2024-09-15T21:25:24.813616+0000","mapping_epoch":47,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14631529,"num_objects":3573,"num_object_clones":0,"num_object_copies":7146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":235463297,"num_objects":57488,"num_object_clones":0,"num_object_copies":114976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210291,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16283,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":636788736,"data_stored":636731886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160677,"ondisk_log_size":160677,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953571,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499892,"kb_used_data":225252,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93871948,"statfs":{"total":96636764160,"available":96124874752,"internally_reserved":0,"allocated":230658048,"data_stored":230356646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149798,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614528,"kb_used_data":226096,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93757312,"statfs":{"total":96636764160,"available":96007487488,"internally_reserved":0,"allocated":231522304,"data_stored":231224974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":397733553},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511336,"kb_used_data":172824,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93860504,"statfs":{"total":96636764160,"available":96113156096,"internally_reserved":0,"allocated":176971776,"data_stored":176632105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":346615468},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230903808,"data_stored":230900329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176078848,"data_stored":176033028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":229806080,"data_stored":229798529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:30.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:31.441 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:31.508 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-15T21:25:30.491323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225448596,"num_objects":55045,"num_object_clones":0,"num_object_copies":110090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212738,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16359,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1610516,"kb_used_data":604756,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281505004,"statfs":{"total":289910292480,"available":288261124096,"internally_reserved":0,"allocated":619270144,"data_stored":618331741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1029819402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29716480,"num_objects":-7255,"num_object_clones":0,"num_object_copies":-14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7255,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009184"},"pg_stats":[{"pgid":"2.f","version":"52'15018","reported_seq":15779,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.328805+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:29.328805+0000","last_peered":"2024-09-15T21:25:29.328805+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:29.328805+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14311424,"num_objects":3494,"num_object_clones":0,"num_object_copies":6988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15018,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15160","reported_seq":15270,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872550+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:29.872550+0000","last_peered":"2024-09-15T21:25:29.872550+0000","last_clean":"2024-09-15T21:25:29.872550+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:29.872550+0000","last_undegraded":"2024-09-15T21:25:29.872550+0000","last_fullsized":"2024-09-15T21:25:29.872550+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14254080,"num_objects":3480,"num_object_clones":0,"num_object_copies":6960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13886,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15267","reported_seq":22197,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.902737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:29.902737+0000","last_peered":"2024-09-15T21:25:29.902737+0000","last_clean":"2024-09-15T21:25:29.902737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:29.902737+0000","last_undegraded":"2024-09-15T21:25:29.902737+0000","last_fullsized":"2024-09-15T21:25:29.902737+0000","mapping_epoch":45,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753816,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13990,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15169","reported_seq":15252,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.836735+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:29.836735+0000","last_peered":"2024-09-15T21:25:29.836735+0000","last_clean":"2024-09-15T21:25:29.836735+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:29.836735+0000","last_undegraded":"2024-09-15T21:25:29.836735+0000","last_fullsized":"2024-09-15T21:25:29.836735+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15169,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15214","reported_seq":15346,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.379702+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:29.379702+0000","last_peered":"2024-09-15T21:25:29.379702+0000","last_clean":"2024-09-15T21:25:29.379702+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:29.379702+0000","last_undegraded":"2024-09-15T21:25:29.379702+0000","last_fullsized":"2024-09-15T21:25:29.379702+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13072,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15003","reported_seq":15830,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.347659+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:29.347659+0000","last_peered":"2024-09-15T21:25:29.347659+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:29.347659+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836288,"num_objects":3378,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15437","reported_seq":16188,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.320448+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:29.320448+0000","last_peered":"2024-09-15T21:25:29.320448+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:29.320448+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14964","reported_seq":15097,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.385545+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:29.385545+0000","last_peered":"2024-09-15T21:25:29.385545+0000","last_clean":"2024-09-15T21:25:29.385545+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:29.385545+0000","last_undegraded":"2024-09-15T21:25:29.385545+0000","last_fullsized":"2024-09-15T21:25:29.385545+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12749,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14947","reported_seq":21774,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.869892+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:29.869892+0000","last_peered":"2024-09-15T21:25:29.869892+0000","last_clean":"2024-09-15T21:25:29.869892+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:29.869892+0000","last_undegraded":"2024-09-15T21:25:29.869892+0000","last_fullsized":"2024-09-15T21:25:29.869892+0000","mapping_epoch":49,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14041088,"num_objects":3428,"num_object_clones":0,"num_object_copies":6856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13533,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14721","reported_seq":15507,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.346657+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:29.346657+0000","last_peered":"2024-09-15T21:25:29.346657+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:29.346657+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14878","reported_seq":16500,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.870790+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:29.870790+0000","last_peered":"2024-09-15T21:25:29.870790+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:29.870790+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13955072,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13429,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":749,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15091","reported_seq":15505,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.885673+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:29.885673+0000","last_peered":"2024-09-15T21:25:29.885673+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:29.885673+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14159872,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15231","reported_seq":15979,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.392978+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:29.392978+0000","last_peered":"2024-09-15T21:25:29.392978+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:29.392978+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14249984,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15012","reported_seq":21930,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.875621+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:29.875621+0000","last_peered":"2024-09-15T21:25:29.875621+0000","last_clean":"2024-09-15T21:25:29.875621+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:29.875621+0000","last_undegraded":"2024-09-15T21:25:29.875621+0000","last_fullsized":"2024-09-15T21:25:29.875621+0000","mapping_epoch":51,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14998","reported_seq":15661,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.373362+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:29.373362+0000","last_peered":"2024-09-15T21:25:29.373362+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:29.373362+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12645,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15212","reported_seq":22310,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872745+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:29.872745+0000","last_peered":"2024-09-15T21:25:29.872745+0000","last_clean":"2024-09-15T21:25:29.872745+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:29.872745+0000","last_undegraded":"2024-09-15T21:25:29.872745+0000","last_fullsized":"2024-09-15T21:25:29.872745+0000","mapping_epoch":47,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951593,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12872,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":225448577,"num_objects":55043,"num_object_clones":0,"num_object_copies":110086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212736,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16358,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":616595456,"data_stored":616538606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486728,"kb_used_data":220408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93885112,"statfs":{"total":96636764160,"available":96138354688,"internally_reserved":0,"allocated":225697792,"data_stored":225396390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611472,"kb_used_data":218868,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93760368,"statfs":{"total":96636764160,"available":96010616832,"internally_reserved":0,"allocated":224120832,"data_stored":223823502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512316,"kb_used_data":165480,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93859524,"statfs":{"total":96636764160,"available":96112152576,"internally_reserved":0,"allocated":169451520,"data_stored":169111849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":223383552,"data_stored":223380073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168407040,"data_stored":168361220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224804864,"data_stored":224797313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:31.509 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:32.257 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:32.257 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:32.303 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-15T21:25:30.491323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225448596,"num_objects":55045,"num_object_clones":0,"num_object_copies":110090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212738,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16359,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1610516,"kb_used_data":604756,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281505004,"statfs":{"total":289910292480,"available":288261124096,"internally_reserved":0,"allocated":619270144,"data_stored":618331741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1029819402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29716480,"num_objects":-7255,"num_object_clones":0,"num_object_copies":-14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7255,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009184"},"pg_stats":[{"pgid":"2.f","version":"52'15018","reported_seq":15779,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.328805+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:29.328805+0000","last_peered":"2024-09-15T21:25:29.328805+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:29.328805+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14311424,"num_objects":3494,"num_object_clones":0,"num_object_copies":6988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15018,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15160","reported_seq":15270,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872550+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:29.872550+0000","last_peered":"2024-09-15T21:25:29.872550+0000","last_clean":"2024-09-15T21:25:29.872550+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:29.872550+0000","last_undegraded":"2024-09-15T21:25:29.872550+0000","last_fullsized":"2024-09-15T21:25:29.872550+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14254080,"num_objects":3480,"num_object_clones":0,"num_object_copies":6960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13886,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15267","reported_seq":22197,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.902737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:29.902737+0000","last_peered":"2024-09-15T21:25:29.902737+0000","last_clean":"2024-09-15T21:25:29.902737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:29.902737+0000","last_undegraded":"2024-09-15T21:25:29.902737+0000","last_fullsized":"2024-09-15T21:25:29.902737+0000","mapping_epoch":45,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753816,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13990,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15169","reported_seq":15252,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.836735+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:29.836735+0000","last_peered":"2024-09-15T21:25:29.836735+0000","last_clean":"2024-09-15T21:25:29.836735+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:29.836735+0000","last_undegraded":"2024-09-15T21:25:29.836735+0000","last_fullsized":"2024-09-15T21:25:29.836735+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15169,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15214","reported_seq":15346,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.379702+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:29.379702+0000","last_peered":"2024-09-15T21:25:29.379702+0000","last_clean":"2024-09-15T21:25:29.379702+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:29.379702+0000","last_undegraded":"2024-09-15T21:25:29.379702+0000","last_fullsized":"2024-09-15T21:25:29.379702+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13072,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15003","reported_seq":15830,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.347659+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:29.347659+0000","last_peered":"2024-09-15T21:25:29.347659+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:29.347659+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836288,"num_objects":3378,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15437","reported_seq":16188,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.320448+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:29.320448+0000","last_peered":"2024-09-15T21:25:29.320448+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:29.320448+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14964","reported_seq":15097,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.385545+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:29.385545+0000","last_peered":"2024-09-15T21:25:29.385545+0000","last_clean":"2024-09-15T21:25:29.385545+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:29.385545+0000","last_undegraded":"2024-09-15T21:25:29.385545+0000","last_fullsized":"2024-09-15T21:25:29.385545+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12749,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14947","reported_seq":21774,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.869892+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:29.869892+0000","last_peered":"2024-09-15T21:25:29.869892+0000","last_clean":"2024-09-15T21:25:29.869892+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:29.869892+0000","last_undegraded":"2024-09-15T21:25:29.869892+0000","last_fullsized":"2024-09-15T21:25:29.869892+0000","mapping_epoch":49,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14041088,"num_objects":3428,"num_object_clones":0,"num_object_copies":6856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13533,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14721","reported_seq":15507,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.346657+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:29.346657+0000","last_peered":"2024-09-15T21:25:29.346657+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:29.346657+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14878","reported_seq":16500,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.870790+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:29.870790+0000","last_peered":"2024-09-15T21:25:29.870790+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:29.870790+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13955072,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13429,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":749,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15091","reported_seq":15505,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.885673+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:29.885673+0000","last_peered":"2024-09-15T21:25:29.885673+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:29.885673+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14159872,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15231","reported_seq":15979,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.392978+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:29.392978+0000","last_peered":"2024-09-15T21:25:29.392978+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:29.392978+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14249984,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15012","reported_seq":21930,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.875621+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:29.875621+0000","last_peered":"2024-09-15T21:25:29.875621+0000","last_clean":"2024-09-15T21:25:29.875621+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:29.875621+0000","last_undegraded":"2024-09-15T21:25:29.875621+0000","last_fullsized":"2024-09-15T21:25:29.875621+0000","mapping_epoch":51,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14998","reported_seq":15661,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.373362+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:29.373362+0000","last_peered":"2024-09-15T21:25:29.373362+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:29.373362+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12645,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15212","reported_seq":22310,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872745+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:29.872745+0000","last_peered":"2024-09-15T21:25:29.872745+0000","last_clean":"2024-09-15T21:25:29.872745+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:29.872745+0000","last_undegraded":"2024-09-15T21:25:29.872745+0000","last_fullsized":"2024-09-15T21:25:29.872745+0000","mapping_epoch":47,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951593,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12872,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":225448577,"num_objects":55043,"num_object_clones":0,"num_object_copies":110086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212736,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16358,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":616595456,"data_stored":616538606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486728,"kb_used_data":220408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93885112,"statfs":{"total":96636764160,"available":96138354688,"internally_reserved":0,"allocated":225697792,"data_stored":225396390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611472,"kb_used_data":218868,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93760368,"statfs":{"total":96636764160,"available":96010616832,"internally_reserved":0,"allocated":224120832,"data_stored":223823502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512316,"kb_used_data":165480,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93859524,"statfs":{"total":96636764160,"available":96112152576,"internally_reserved":0,"allocated":169451520,"data_stored":169111849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":223383552,"data_stored":223380073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168407040,"data_stored":168361220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224804864,"data_stored":224797313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15018", 'reported_seq': 15779, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.328805+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:29.328805+0000', 'last_peered': '2024-09-15T21:25:29.328805+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:29.328805+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14311424, 'num_objects': 3494, 'num_object_clones': 0, 'num_object_copies': 6988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15018, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 463}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15003", 'reported_seq': 15830, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.347659+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:29.347659+0000', 'last_peered': '2024-09-15T21:25:29.347659+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:29.347659+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13836288, 'num_objects': 3378, 'num_object_clones': 0, 'num_object_copies': 6756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12428, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'15437", 'reported_seq': 16188, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.320448+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:29.320448+0000', 'last_peered': '2024-09-15T21:25:29.320448+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:29.320448+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14200832, 'num_objects': 3467, 'num_object_clones': 0, 'num_object_copies': 6934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13073, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14721", 'reported_seq': 15507, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.346657+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:29.346657+0000', 'last_peered': '2024-09-15T21:25:29.346657+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:29.346657+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4700", 'ondisk_log_start': "44'4700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13570048, 'num_objects': 3313, 'num_object_clones': 0, 'num_object_copies': 6626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12247, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 298}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'14878", 'reported_seq': 16500, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.870790+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:29.870790+0000', 'last_peered': '2024-09-15T21:25:29.870790+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:29.870790+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13955072, 'num_objects': 3407, 'num_object_clones': 0, 'num_object_copies': 6814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2519, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13429, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 749, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 888}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15091", 'reported_seq': 15505, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.885673+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:29.885673+0000', 'last_peered': '2024-09-15T21:25:29.885673+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:29.885673+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14159872, 'num_objects': 3457, 'num_object_clones': 0, 'num_object_copies': 6914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13490, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 251}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15231", 'reported_seq': 15979, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.392978+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:29.392978+0000', 'last_peered': '2024-09-15T21:25:29.392978+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:29.392978+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14249984, 'num_objects': 3479, 'num_object_clones': 0, 'num_object_copies': 6958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12538, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:32.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'14998", 'reported_seq': 15661, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:29.373362+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:29.373362+0000', 'last_peered': '2024-09-15T21:25:29.373362+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:29.373362+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'4900", 'ondisk_log_start': "44'4900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13844480, 'num_objects': 3380, 'num_object_clones': 0, 'num_object_copies': 6760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12645, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:32.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:33.287 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:33.287 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24207/110090 objects degraded (21.988%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":546,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":55045,"data_bytes":225448596,"bytes_used":1649168384,"bytes_avail":288261124096,"bytes_total":289910292480,"degraded_objects":24207,"degraded_total":110090,"degraded_ratio":0.21988373149241527,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":161,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:33.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:33.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:33.881 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:33.882 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:33.929 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-15T21:25:32.491796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":225448596,"num_objects":55045,"num_object_clones":0,"num_object_copies":110090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212738,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16359,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1610516,"kb_used_data":604756,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281505004,"statfs":{"total":289910292480,"available":288261124096,"internally_reserved":0,"allocated":619270144,"data_stored":618331741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1029819402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20230144,"num_objects":-4939,"num_object_clones":0,"num_object_copies":-9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4939,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008310"},"pg_stats":[{"pgid":"2.f","version":"52'15018","reported_seq":15779,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.328805+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:29.328805+0000","last_peered":"2024-09-15T21:25:29.328805+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:29.328805+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14311424,"num_objects":3494,"num_object_clones":0,"num_object_copies":6988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15018,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15160","reported_seq":15270,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872550+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:29.872550+0000","last_peered":"2024-09-15T21:25:29.872550+0000","last_clean":"2024-09-15T21:25:29.872550+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:29.872550+0000","last_undegraded":"2024-09-15T21:25:29.872550+0000","last_fullsized":"2024-09-15T21:25:29.872550+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14254080,"num_objects":3480,"num_object_clones":0,"num_object_copies":6960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13886,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15267","reported_seq":22197,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.902737+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:29.902737+0000","last_peered":"2024-09-15T21:25:29.902737+0000","last_clean":"2024-09-15T21:25:29.902737+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:29.902737+0000","last_undegraded":"2024-09-15T21:25:29.902737+0000","last_fullsized":"2024-09-15T21:25:29.902737+0000","mapping_epoch":45,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753816,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13990,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15169","reported_seq":15252,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.836735+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:29.836735+0000","last_peered":"2024-09-15T21:25:29.836735+0000","last_clean":"2024-09-15T21:25:29.836735+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:29.836735+0000","last_undegraded":"2024-09-15T21:25:29.836735+0000","last_fullsized":"2024-09-15T21:25:29.836735+0000","mapping_epoch":36,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930496,"num_objects":3401,"num_object_clones":0,"num_object_copies":6802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15169,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15214","reported_seq":15346,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.379702+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:29.379702+0000","last_peered":"2024-09-15T21:25:29.379702+0000","last_clean":"2024-09-15T21:25:29.379702+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:29.379702+0000","last_undegraded":"2024-09-15T21:25:29.379702+0000","last_fullsized":"2024-09-15T21:25:29.379702+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14372864,"num_objects":3509,"num_object_clones":0,"num_object_copies":7018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13072,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15003","reported_seq":15830,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.347659+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:29.347659+0000","last_peered":"2024-09-15T21:25:29.347659+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:29.347659+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836288,"num_objects":3378,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15437","reported_seq":16188,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.320448+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:29.320448+0000","last_peered":"2024-09-15T21:25:29.320448+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:29.320448+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'14964","reported_seq":15097,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.385545+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:29.385545+0000","last_peered":"2024-09-15T21:25:29.385545+0000","last_clean":"2024-09-15T21:25:29.385545+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:29.385545+0000","last_undegraded":"2024-09-15T21:25:29.385545+0000","last_fullsized":"2024-09-15T21:25:29.385545+0000","mapping_epoch":35,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12749,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'14947","reported_seq":21774,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.869892+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:29.869892+0000","last_peered":"2024-09-15T21:25:29.869892+0000","last_clean":"2024-09-15T21:25:29.869892+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:29.869892+0000","last_undegraded":"2024-09-15T21:25:29.869892+0000","last_fullsized":"2024-09-15T21:25:29.869892+0000","mapping_epoch":49,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14041088,"num_objects":3428,"num_object_clones":0,"num_object_copies":6856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13533,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14721","reported_seq":15507,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.346657+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:29.346657+0000","last_peered":"2024-09-15T21:25:29.346657+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:29.346657+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4700","ondisk_log_start":"44'4700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12247,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'14878","reported_seq":16500,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.870790+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:29.870790+0000","last_peered":"2024-09-15T21:25:29.870790+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:29.870790+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13955072,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13429,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":749,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15091","reported_seq":15505,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.885673+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:29.885673+0000","last_peered":"2024-09-15T21:25:29.885673+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:29.885673+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14159872,"num_objects":3457,"num_object_clones":0,"num_object_copies":6914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15231","reported_seq":15979,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.392978+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:29.392978+0000","last_peered":"2024-09-15T21:25:29.392978+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:29.392978+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14249984,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12538,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15012","reported_seq":21930,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.875621+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:29.875621+0000","last_peered":"2024-09-15T21:25:29.875621+0000","last_clean":"2024-09-15T21:25:29.875621+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:29.875621+0000","last_undegraded":"2024-09-15T21:25:29.875621+0000","last_fullsized":"2024-09-15T21:25:29.875621+0000","mapping_epoch":51,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13926400,"num_objects":3400,"num_object_clones":0,"num_object_copies":6800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'14998","reported_seq":15661,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:29.373362+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:29.373362+0000","last_peered":"2024-09-15T21:25:29.373362+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:29.373362+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'4900","ondisk_log_start":"44'4900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12645,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15212","reported_seq":22310,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:29.872745+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:29.872745+0000","last_peered":"2024-09-15T21:25:29.872745+0000","last_clean":"2024-09-15T21:25:29.872745+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:29.872745+0000","last_undegraded":"2024-09-15T21:25:29.872745+0000","last_fullsized":"2024-09-15T21:25:29.872745+0000","mapping_epoch":47,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13951593,"num_objects":3407,"num_object_clones":0,"num_object_copies":6814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12872,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":225448577,"num_objects":55043,"num_object_clones":0,"num_object_copies":110086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212736,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16358,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":616595456,"data_stored":616538606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953572,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486728,"kb_used_data":220408,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93885112,"statfs":{"total":96636764160,"available":96138354688,"internally_reserved":0,"allocated":225697792,"data_stored":225396390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149799,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611472,"kb_used_data":218868,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93760368,"statfs":{"total":96636764160,"available":96010616832,"internally_reserved":0,"allocated":224120832,"data_stored":223823502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512316,"kb_used_data":165480,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93859524,"statfs":{"total":96636764160,"available":96112152576,"internally_reserved":0,"allocated":169451520,"data_stored":169111849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":355135148},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":223383552,"data_stored":223380073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168407040,"data_stored":168361220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224804864,"data_stored":224797313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:36.932 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:37.658 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:37.659 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:37.723 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-15T21:25:36.492671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215933588,"num_objects":52722,"num_object_clones":0,"num_object_copies":105444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215061,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16482,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609868,"kb_used_data":586068,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281505652,"statfs":{"total":289910292480,"available":288261787648,"internally_reserved":0,"allocated":600133632,"data_stored":599195229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1048300554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24813568,"num_objects":-6058,"num_object_clones":0,"num_object_copies":-12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6058,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006975"},"pg_stats":[{"pgid":"2.f","version":"52'15170","reported_seq":15932,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.373357+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:34.373357+0000","last_peered":"2024-09-15T21:25:34.373357+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:34.373357+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688832,"num_objects":3342,"num_object_clones":0,"num_object_copies":6684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15294","reported_seq":15404,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.900399+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:34.900399+0000","last_peered":"2024-09-15T21:25:34.900399+0000","last_clean":"2024-09-15T21:25:34.900399+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:34.900399+0000","last_undegraded":"2024-09-15T21:25:34.900399+0000","last_fullsized":"2024-09-15T21:25:34.900399+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13705216,"num_objects":3346,"num_object_clones":0,"num_object_copies":6692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14020,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15420","reported_seq":22350,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.904790+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:34.904790+0000","last_peered":"2024-09-15T21:25:34.904790+0000","last_clean":"2024-09-15T21:25:34.904790+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:34.904790+0000","last_undegraded":"2024-09-15T21:25:34.904790+0000","last_fullsized":"2024-09-15T21:25:34.904790+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14127128,"num_objects":3450,"num_object_clones":0,"num_object_copies":6900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14143,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15323","reported_seq":15406,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.822277+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:34.822277+0000","last_peered":"2024-09-15T21:25:34.822277+0000","last_clean":"2024-09-15T21:25:34.822277+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:34.822277+0000","last_undegraded":"2024-09-15T21:25:34.822277+0000","last_fullsized":"2024-09-15T21:25:34.822277+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13299712,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15323,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15367","reported_seq":15499,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.375043+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:34.375043+0000","last_peered":"2024-09-15T21:25:34.375043+0000","last_clean":"2024-09-15T21:25:34.375043+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:34.375043+0000","last_undegraded":"2024-09-15T21:25:34.375043+0000","last_fullsized":"2024-09-15T21:25:34.375043+0000","mapping_epoch":35,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13225,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15149","reported_seq":15976,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.338831+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:34.338831+0000","last_peered":"2024-09-15T21:25:34.338831+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:34.338831+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13238272,"num_objects":3232,"num_object_clones":0,"num_object_copies":6464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12574,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15581","reported_seq":16326,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.407177+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:34.407177+0000","last_peered":"2024-09-15T21:25:34.407177+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:34.407177+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13217,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15135","reported_seq":15268,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.390923+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:34.390923+0000","last_peered":"2024-09-15T21:25:34.390923+0000","last_clean":"2024-09-15T21:25:34.390923+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:34.390923+0000","last_undegraded":"2024-09-15T21:25:34.390923+0000","last_fullsized":"2024-09-15T21:25:34.390923+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15076","reported_seq":21903,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.877125+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:34.877125+0000","last_peered":"2024-09-15T21:25:34.877125+0000","last_clean":"2024-09-15T21:25:34.877125+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:34.877125+0000","last_undegraded":"2024-09-15T21:25:34.877125+0000","last_fullsized":"2024-09-15T21:25:34.877125+0000","mapping_epoch":49,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13512704,"num_objects":3299,"num_object_clones":0,"num_object_copies":6598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14876","reported_seq":15664,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.339448+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:34.339448+0000","last_peered":"2024-09-15T21:25:34.339448+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:34.339448+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12402,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":143}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15008","reported_seq":16875,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.872059+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:34.872059+0000","last_peered":"2024-09-15T21:25:34.872059+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:34.872059+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2396,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13559,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":872,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15234","reported_seq":15650,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.876548+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:34.876548+0000","last_peered":"2024-09-15T21:25:34.876548+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:34.876548+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15379","reported_seq":16127,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.303779+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:34.303779+0000","last_peered":"2024-09-15T21:25:34.303779+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:34.303779+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13643776,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15142","reported_seq":22057,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.937544+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:34.937544+0000","last_peered":"2024-09-15T21:25:34.937544+0000","last_clean":"2024-09-15T21:25:34.937544+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:34.937544+0000","last_undegraded":"2024-09-15T21:25:34.937544+0000","last_fullsized":"2024-09-15T21:25:34.937544+0000","mapping_epoch":51,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13393920,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12727,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15128","reported_seq":15791,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.369739+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:34.369739+0000","last_peered":"2024-09-15T21:25:34.369739+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:34.369739+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13312000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12775,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15363","reported_seq":22461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.953366+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:34.953366+0000","last_peered":"2024-09-15T21:25:34.953366+0000","last_clean":"2024-09-15T21:25:34.953366+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:34.953366+0000","last_undegraded":"2024-09-15T21:25:34.953366+0000","last_fullsized":"2024-09-15T21:25:34.953366+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13333097,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13023,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":215933569,"num_objects":52720,"num_object_clones":0,"num_object_copies":105440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215059,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16481,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":597639168,"data_stored":597582318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953573,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504128,"kb_used_data":215592,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93867712,"statfs":{"total":96636764160,"available":96120537088,"internally_reserved":0,"allocated":220766208,"data_stored":220464806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149800,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612752,"kb_used_data":211840,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93759088,"statfs":{"total":96636764160,"available":96009306112,"internally_reserved":0,"allocated":216924160,"data_stored":216626830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":410513073},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492988,"kb_used_data":158636,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878852,"statfs":{"total":96636764160,"available":96131944448,"internally_reserved":0,"allocated":162443264,"data_stored":162103593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":216223744,"data_stored":216220265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":161374208,"data_stored":161328388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":220041216,"data_stored":220033665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:37.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:38.567 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:38.618 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-15T21:25:38.493150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215933588,"num_objects":52722,"num_object_clones":0,"num_object_copies":105444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215061,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16482,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609868,"kb_used_data":586068,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281505652,"statfs":{"total":289910292480,"available":288261787648,"internally_reserved":0,"allocated":600133632,"data_stored":599195229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1048300554},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19529728,"num_objects":-4768,"num_object_clones":0,"num_object_copies":-9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007040"},"pg_stats":[{"pgid":"2.f","version":"52'15170","reported_seq":15932,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.373357+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:34.373357+0000","last_peered":"2024-09-15T21:25:34.373357+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:34.373357+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688832,"num_objects":3342,"num_object_clones":0,"num_object_copies":6684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15294","reported_seq":15404,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.900399+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:34.900399+0000","last_peered":"2024-09-15T21:25:34.900399+0000","last_clean":"2024-09-15T21:25:34.900399+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:34.900399+0000","last_undegraded":"2024-09-15T21:25:34.900399+0000","last_fullsized":"2024-09-15T21:25:34.900399+0000","mapping_epoch":36,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13705216,"num_objects":3346,"num_object_clones":0,"num_object_copies":6692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14020,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15420","reported_seq":22350,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.904790+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:34.904790+0000","last_peered":"2024-09-15T21:25:34.904790+0000","last_clean":"2024-09-15T21:25:34.904790+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:34.904790+0000","last_undegraded":"2024-09-15T21:25:34.904790+0000","last_fullsized":"2024-09-15T21:25:34.904790+0000","mapping_epoch":45,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14127128,"num_objects":3450,"num_object_clones":0,"num_object_copies":6900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14143,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15323","reported_seq":15406,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.822277+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:34.822277+0000","last_peered":"2024-09-15T21:25:34.822277+0000","last_clean":"2024-09-15T21:25:34.822277+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:34.822277+0000","last_undegraded":"2024-09-15T21:25:34.822277+0000","last_fullsized":"2024-09-15T21:25:34.822277+0000","mapping_epoch":36,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13299712,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15323,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15367","reported_seq":15499,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.375043+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:34.375043+0000","last_peered":"2024-09-15T21:25:34.375043+0000","last_clean":"2024-09-15T21:25:34.375043+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:34.375043+0000","last_undegraded":"2024-09-15T21:25:34.375043+0000","last_fullsized":"2024-09-15T21:25:34.375043+0000","mapping_epoch":35,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13225,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15149","reported_seq":15976,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.338831+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:34.338831+0000","last_peered":"2024-09-15T21:25:34.338831+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:34.338831+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13238272,"num_objects":3232,"num_object_clones":0,"num_object_copies":6464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12574,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15581","reported_seq":16326,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.407177+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:34.407177+0000","last_peered":"2024-09-15T21:25:34.407177+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:34.407177+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13217,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15135","reported_seq":15268,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.390923+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:34.390923+0000","last_peered":"2024-09-15T21:25:34.390923+0000","last_clean":"2024-09-15T21:25:34.390923+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:34.390923+0000","last_undegraded":"2024-09-15T21:25:34.390923+0000","last_fullsized":"2024-09-15T21:25:34.390923+0000","mapping_epoch":35,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389824,"num_objects":3269,"num_object_clones":0,"num_object_copies":6538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12920,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15076","reported_seq":21903,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.877125+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:34.877125+0000","last_peered":"2024-09-15T21:25:34.877125+0000","last_clean":"2024-09-15T21:25:34.877125+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:34.877125+0000","last_undegraded":"2024-09-15T21:25:34.877125+0000","last_fullsized":"2024-09-15T21:25:34.877125+0000","mapping_epoch":49,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13512704,"num_objects":3299,"num_object_clones":0,"num_object_copies":6598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'14876","reported_seq":15664,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.339448+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:34.339448+0000","last_peered":"2024-09-15T21:25:34.339448+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:34.339448+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'4800","ondisk_log_start":"44'4800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12402,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":143}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15008","reported_seq":16875,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.872059+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:34.872059+0000","last_peered":"2024-09-15T21:25:34.872059+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:34.872059+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2396,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13559,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":872,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15234","reported_seq":15650,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.876548+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:34.876548+0000","last_peered":"2024-09-15T21:25:34.876548+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:34.876548+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13574144,"num_objects":3314,"num_object_clones":0,"num_object_copies":6628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15379","reported_seq":16127,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.303779+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:34.303779+0000","last_peered":"2024-09-15T21:25:34.303779+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:34.303779+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13643776,"num_objects":3331,"num_object_clones":0,"num_object_copies":6662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15142","reported_seq":22057,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.937544+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:34.937544+0000","last_peered":"2024-09-15T21:25:34.937544+0000","last_clean":"2024-09-15T21:25:34.937544+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:34.937544+0000","last_undegraded":"2024-09-15T21:25:34.937544+0000","last_fullsized":"2024-09-15T21:25:34.937544+0000","mapping_epoch":51,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13393920,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12727,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15128","reported_seq":15791,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:34.369739+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:34.369739+0000","last_peered":"2024-09-15T21:25:34.369739+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:34.369739+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13312000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12775,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15363","reported_seq":22461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:34.953366+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:34.953366+0000","last_peered":"2024-09-15T21:25:34.953366+0000","last_clean":"2024-09-15T21:25:34.953366+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:34.953366+0000","last_undegraded":"2024-09-15T21:25:34.953366+0000","last_fullsized":"2024-09-15T21:25:34.953366+0000","mapping_epoch":47,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13333097,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13023,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":215933569,"num_objects":52720,"num_object_clones":0,"num_object_copies":105440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215059,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16481,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":597639168,"data_stored":597582318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953573,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504128,"kb_used_data":215592,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93867712,"statfs":{"total":96636764160,"available":96120537088,"internally_reserved":0,"allocated":220766208,"data_stored":220464806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149800,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612752,"kb_used_data":211840,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93759088,"statfs":{"total":96636764160,"available":96009306112,"internally_reserved":0,"allocated":216924160,"data_stored":216626830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":410513073},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626519,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492988,"kb_used_data":158636,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878852,"statfs":{"total":96636764160,"available":96131944448,"internally_reserved":0,"allocated":162443264,"data_stored":162103593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":342355628},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":216223744,"data_stored":216220265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":161374208,"data_stored":161328388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":220041216,"data_stored":220033665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15170", 'reported_seq': 15932, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.373357+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:34.373357+0000', 'last_peered': '2024-09-15T21:25:34.373357+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:34.373357+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13688832, 'num_objects': 3342, 'num_object_clones': 0, 'num_object_copies': 6684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15170, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 311}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15149", 'reported_seq': 15976, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.338831+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:34.338831+0000', 'last_peered': '2024-09-15T21:25:34.338831+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:34.338831+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13238272, 'num_objects': 3232, 'num_object_clones': 0, 'num_object_copies': 6464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3232, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12574, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'15581", 'reported_seq': 16326, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.407177+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:34.407177+0000', 'last_peered': '2024-09-15T21:25:34.407177+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:34.407177+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13611008, 'num_objects': 3323, 'num_object_clones': 0, 'num_object_copies': 6646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13217, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:38.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'14876", 'reported_seq': 15664, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.339448+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:34.339448+0000', 'last_peered': '2024-09-15T21:25:34.339448+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:34.339448+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'4800", 'ondisk_log_start': "44'4800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12935168, 'num_objects': 3158, 'num_object_clones': 0, 'num_object_copies': 6316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12402, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 143}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'15008", 'reported_seq': 16875, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.872059+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:34.872059+0000', 'last_peered': '2024-09-15T21:25:34.872059+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:34.872059+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'5000", 'ondisk_log_start': "44'5000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13422592, 'num_objects': 3277, 'num_object_clones': 0, 'num_object_copies': 6554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2396, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13559, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 872, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 881}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15234", 'reported_seq': 15650, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.876548+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:34.876548+0000', 'last_peered': '2024-09-15T21:25:34.876548+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:34.876548+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5200", 'ondisk_log_start': "44'5200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13574144, 'num_objects': 3314, 'num_object_clones': 0, 'num_object_copies': 6628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13633, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 108}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15379", 'reported_seq': 16127, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.303779+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:34.303779+0000', 'last_peered': '2024-09-15T21:25:34.303779+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:34.303779+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13643776, 'num_objects': 3331, 'num_object_clones': 0, 'num_object_copies': 6662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12686, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:38.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'15128", 'reported_seq': 15791, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:34.369739+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:34.369739+0000', 'last_peered': '2024-09-15T21:25:34.369739+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:34.369739+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13312000, 'num_objects': 3250, 'num_object_clones': 0, 'num_object_copies': 6500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12775, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:38.621 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:39.502 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:39.502 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24084/105444 objects degraded (22.841%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":552,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":52722,"data_bytes":215933588,"bytes_used":1648504832,"bytes_avail":288261787648,"bytes_total":289910292480,"degraded_objects":24084,"degraded_total":105444,"degraded_ratio":0.22840559918060768,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:39.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:39.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:40.533 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:40.573 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-15T21:25:40.497044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206123668,"num_objects":50327,"num_object_clones":0,"num_object_copies":100654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217456,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608836,"kb_used_data":566996,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281506684,"statfs":{"total":289910292480,"available":288262844416,"internally_reserved":0,"allocated":580603904,"data_stored":579665501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1066781706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,3,10,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29339648,"num_objects":-7163,"num_object_clones":0,"num_object_copies":-14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7163,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006121"},"pg_stats":[{"pgid":"2.f","version":"52'15319","reported_seq":16080,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.379897+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:39.379897+0000","last_peered":"2024-09-15T21:25:39.379897+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:39.379897+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13078528,"num_objects":3193,"num_object_clones":0,"num_object_copies":6386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15319,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15449","reported_seq":15559,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.832868+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:39.832868+0000","last_peered":"2024-09-15T21:25:39.832868+0000","last_clean":"2024-09-15T21:25:39.832868+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:39.832868+0000","last_undegraded":"2024-09-15T21:25:39.832868+0000","last_fullsized":"2024-09-15T21:25:39.832868+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15566","reported_seq":22496,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.901659+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:39.901659+0000","last_peered":"2024-09-15T21:25:39.901659+0000","last_clean":"2024-09-15T21:25:39.901659+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:39.901659+0000","last_undegraded":"2024-09-15T21:25:39.901659+0000","last_fullsized":"2024-09-15T21:25:39.901659+0000","mapping_epoch":45,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13529112,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14289,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15486","reported_seq":15569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.825929+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:39.825929+0000","last_peered":"2024-09-15T21:25:39.825929+0000","last_clean":"2024-09-15T21:25:39.825929+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:39.825929+0000","last_undegraded":"2024-09-15T21:25:39.825929+0000","last_fullsized":"2024-09-15T21:25:39.825929+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15518","reported_seq":15650,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.403865+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:39.403865+0000","last_peered":"2024-09-15T21:25:39.403865+0000","last_clean":"2024-09-15T21:25:39.403865+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:39.403865+0000","last_undegraded":"2024-09-15T21:25:39.403865+0000","last_fullsized":"2024-09-15T21:25:39.403865+0000","mapping_epoch":35,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13127680,"num_objects":3205,"num_object_clones":0,"num_object_copies":6410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15299","reported_seq":16126,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.284959+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:39.284959+0000","last_peered":"2024-09-15T21:25:39.284959+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:39.284959+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15742","reported_seq":16492,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.346582+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:39.346582+0000","last_peered":"2024-09-15T21:25:39.346582+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:39.346582+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13378,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15282","reported_seq":15415,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.389909+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:39.389909+0000","last_peered":"2024-09-15T21:25:39.389909+0000","last_clean":"2024-09-15T21:25:39.389909+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:39.389909+0000","last_undegraded":"2024-09-15T21:25:39.389909+0000","last_fullsized":"2024-09-15T21:25:39.389909+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12787712,"num_objects":3122,"num_object_clones":0,"num_object_copies":6244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15228","reported_seq":22056,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.994843+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:39.994843+0000","last_peered":"2024-09-15T21:25:39.994843+0000","last_clean":"2024-09-15T21:25:39.994843+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:39.994843+0000","last_undegraded":"2024-09-15T21:25:39.994843+0000","last_fullsized":"2024-09-15T21:25:39.994843+0000","mapping_epoch":49,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15026","reported_seq":15813,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.425653+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:39.425653+0000","last_peered":"2024-09-15T21:25:39.425653+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:39.425653+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320768,"num_objects":3008,"num_object_clones":0,"num_object_copies":6016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12552,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15153","reported_seq":17225,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.884858+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:39.884858+0000","last_peered":"2024-09-15T21:25:39.884858+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:39.884858+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13704,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15370","reported_seq":15786,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.863024+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:39.863024+0000","last_peered":"2024-09-15T21:25:39.863024+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:39.863024+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15532","reported_seq":16278,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.328859+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:39.328859+0000","last_peered":"2024-09-15T21:25:39.328859+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:39.328859+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12839,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15268","reported_seq":22186,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.877835+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:39.877835+0000","last_peered":"2024-09-15T21:25:39.877835+0000","last_clean":"2024-09-15T21:25:39.877835+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:39.877835+0000","last_undegraded":"2024-09-15T21:25:39.877835+0000","last_fullsized":"2024-09-15T21:25:39.877835+0000","mapping_epoch":51,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15279","reported_seq":15942,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.377409+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:39.377409+0000","last_peered":"2024-09-15T21:25:39.377409+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:39.377409+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12693504,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12926,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15523","reported_seq":22621,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.958922+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:39.958922+0000","last_peered":"2024-09-15T21:25:39.958922+0000","last_clean":"2024-09-15T21:25:39.958922+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:39.958922+0000","last_undegraded":"2024-09-15T21:25:39.958922+0000","last_fullsized":"2024-09-15T21:25:39.958922+0000","mapping_epoch":47,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12677737,"num_objects":3096,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":206123649,"num_objects":50325,"num_object_clones":0,"num_object_copies":100650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217454,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16583,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577970176,"data_stored":577913326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953574,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503408,"kb_used_data":210720,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93868432,"statfs":{"total":96636764160,"available":96121274368,"internally_reserved":0,"allocated":215777280,"data_stored":215475878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149801,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592992,"kb_used_data":204560,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93778848,"statfs":{"total":96636764160,"available":96029540352,"internally_reserved":0,"allocated":209469440,"data_stored":209172110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":397733553},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512436,"kb_used_data":151716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93859404,"statfs":{"total":96636764160,"available":96112029696,"internally_reserved":0,"allocated":155357184,"data_stored":155017513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208732160,"data_stored":208728681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154103808,"data_stored":154057988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215134208,"data_stored":215126657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:43.576 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:44.384 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:44.397 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-15T21:25:42.497562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":206123668,"num_objects":50327,"num_object_clones":0,"num_object_copies":100654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217456,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608836,"kb_used_data":566996,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281506684,"statfs":{"total":289910292480,"available":288262844416,"internally_reserved":0,"allocated":580603904,"data_stored":579665501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1066781706},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,3,10,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19324928,"num_objects":-4718,"num_object_clones":0,"num_object_copies":-9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006238"},"pg_stats":[{"pgid":"2.f","version":"52'15319","reported_seq":16080,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.379897+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:39.379897+0000","last_peered":"2024-09-15T21:25:39.379897+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:39.379897+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13078528,"num_objects":3193,"num_object_clones":0,"num_object_copies":6386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15319,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15449","reported_seq":15559,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.832868+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:39.832868+0000","last_peered":"2024-09-15T21:25:39.832868+0000","last_clean":"2024-09-15T21:25:39.832868+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:39.832868+0000","last_undegraded":"2024-09-15T21:25:39.832868+0000","last_fullsized":"2024-09-15T21:25:39.832868+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15566","reported_seq":22496,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.901659+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:39.901659+0000","last_peered":"2024-09-15T21:25:39.901659+0000","last_clean":"2024-09-15T21:25:39.901659+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:39.901659+0000","last_undegraded":"2024-09-15T21:25:39.901659+0000","last_fullsized":"2024-09-15T21:25:39.901659+0000","mapping_epoch":45,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13529112,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14289,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15486","reported_seq":15569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.825929+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:39.825929+0000","last_peered":"2024-09-15T21:25:39.825929+0000","last_clean":"2024-09-15T21:25:39.825929+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:39.825929+0000","last_undegraded":"2024-09-15T21:25:39.825929+0000","last_fullsized":"2024-09-15T21:25:39.825929+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15518","reported_seq":15650,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.403865+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:39.403865+0000","last_peered":"2024-09-15T21:25:39.403865+0000","last_clean":"2024-09-15T21:25:39.403865+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:39.403865+0000","last_undegraded":"2024-09-15T21:25:39.403865+0000","last_fullsized":"2024-09-15T21:25:39.403865+0000","mapping_epoch":35,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13127680,"num_objects":3205,"num_object_clones":0,"num_object_copies":6410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15299","reported_seq":16126,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.284959+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:39.284959+0000","last_peered":"2024-09-15T21:25:39.284959+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:39.284959+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":53}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15742","reported_seq":16492,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.346582+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:39.346582+0000","last_peered":"2024-09-15T21:25:39.346582+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:39.346582+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12951552,"num_objects":3162,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13378,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15282","reported_seq":15415,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.389909+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:39.389909+0000","last_peered":"2024-09-15T21:25:39.389909+0000","last_clean":"2024-09-15T21:25:39.389909+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:39.389909+0000","last_undegraded":"2024-09-15T21:25:39.389909+0000","last_fullsized":"2024-09-15T21:25:39.389909+0000","mapping_epoch":35,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12787712,"num_objects":3122,"num_object_clones":0,"num_object_copies":6244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13067,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15228","reported_seq":22056,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.994843+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:39.994843+0000","last_peered":"2024-09-15T21:25:39.994843+0000","last_clean":"2024-09-15T21:25:39.994843+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:39.994843+0000","last_undegraded":"2024-09-15T21:25:39.994843+0000","last_fullsized":"2024-09-15T21:25:39.994843+0000","mapping_epoch":49,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15026","reported_seq":15813,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.425653+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:39.425653+0000","last_peered":"2024-09-15T21:25:39.425653+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:39.425653+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5000","ondisk_log_start":"44'5000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320768,"num_objects":3008,"num_object_clones":0,"num_object_copies":6016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12552,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-7}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":114,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725057+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725057+0000","last_peered":"2024-09-15T21:25:03.725057+0000","last_clean":"2024-09-15T21:25:03.725057+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725057+0000","last_undegraded":"2024-09-15T21:25:03.725057+0000","last_fullsized":"2024-09-15T21:25:03.725057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15153","reported_seq":17225,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.884858+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:39.884858+0000","last_peered":"2024-09-15T21:25:39.884858+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:39.884858+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13704,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15370","reported_seq":15786,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.863024+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:39.863024+0000","last_peered":"2024-09-15T21:25:39.863024+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:39.863024+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15532","reported_seq":16278,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.328859+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:39.328859+0000","last_peered":"2024-09-15T21:25:39.328859+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:39.328859+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12839,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15268","reported_seq":22186,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.877835+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:39.877835+0000","last_peered":"2024-09-15T21:25:39.877835+0000","last_clean":"2024-09-15T21:25:39.877835+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:39.877835+0000","last_undegraded":"2024-09-15T21:25:39.877835+0000","last_fullsized":"2024-09-15T21:25:39.877835+0000","mapping_epoch":51,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15279","reported_seq":15942,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.377409+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:39.377409+0000","last_peered":"2024-09-15T21:25:39.377409+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:39.377409+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12693504,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12926,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15523","reported_seq":22621,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.958922+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:39.958922+0000","last_peered":"2024-09-15T21:25:39.958922+0000","last_clean":"2024-09-15T21:25:39.958922+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:39.958922+0000","last_undegraded":"2024-09-15T21:25:39.958922+0000","last_fullsized":"2024-09-15T21:25:39.958922+0000","mapping_epoch":47,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12677737,"num_objects":3096,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":206123649,"num_objects":50325,"num_object_clones":0,"num_object_copies":100650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":217454,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16583,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577970176,"data_stored":577913326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953574,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503408,"kb_used_data":210720,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93868432,"statfs":{"total":96636764160,"available":96121274368,"internally_reserved":0,"allocated":215777280,"data_stored":215475878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149801,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592992,"kb_used_data":204560,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93778848,"statfs":{"total":96636764160,"available":96029540352,"internally_reserved":0,"allocated":209469440,"data_stored":209172110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":397733553},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512436,"kb_used_data":151716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93859404,"statfs":{"total":96636764160,"available":96112029696,"internally_reserved":0,"allocated":155357184,"data_stored":155017513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208732160,"data_stored":208728681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154103808,"data_stored":154057988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215134208,"data_stored":215126657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:44.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:44.898 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:44.898 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:44.998 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-15T21:25:44.497975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":201200276,"num_objects":49125,"num_object_clones":0,"num_object_copies":98250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49125,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218658,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16584,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160945,"ondisk_log_size":160945,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1610024,"kb_used_data":559856,"kb_used_omap":12,"kb_used_meta":1050099,"kb_avail":281505496,"statfs":{"total":289910292480,"available":288261627904,"internally_reserved":0,"allocated":573292544,"data_stored":572354141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1075301386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24248320,"num_objects":-5920,"num_object_clones":0,"num_object_copies":-11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5920,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006179"},"pg_stats":[{"pgid":"2.f","version":"52'15468","reported_seq":16230,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.362568+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:44.362568+0000","last_peered":"2024-09-15T21:25:44.362568+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:44.362568+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468224,"num_objects":3044,"num_object_clones":0,"num_object_copies":6088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":13}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15449","reported_seq":15559,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.832868+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:39.832868+0000","last_peered":"2024-09-15T21:25:39.832868+0000","last_clean":"2024-09-15T21:25:39.832868+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:39.832868+0000","last_undegraded":"2024-09-15T21:25:39.832868+0000","last_fullsized":"2024-09-15T21:25:39.832868+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15566","reported_seq":22496,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.901659+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:39.901659+0000","last_peered":"2024-09-15T21:25:39.901659+0000","last_clean":"2024-09-15T21:25:39.901659+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:39.901659+0000","last_undegraded":"2024-09-15T21:25:39.901659+0000","last_fullsized":"2024-09-15T21:25:39.901659+0000","mapping_epoch":45,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13529112,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14289,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15486","reported_seq":15569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.825929+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:39.825929+0000","last_peered":"2024-09-15T21:25:39.825929+0000","last_clean":"2024-09-15T21:25:39.825929+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:39.825929+0000","last_undegraded":"2024-09-15T21:25:39.825929+0000","last_fullsized":"2024-09-15T21:25:39.825929+0000","mapping_epoch":36,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15486,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15673","reported_seq":15805,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.325697+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:44.325697+0000","last_peered":"2024-09-15T21:25:44.325697+0000","last_clean":"2024-09-15T21:25:44.325697+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:44.325697+0000","last_undegraded":"2024-09-15T21:25:44.325697+0000","last_fullsized":"2024-09-15T21:25:44.325697+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13531,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15446","reported_seq":16273,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.327676+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:44.327676+0000","last_peered":"2024-09-15T21:25:44.327676+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:44.327676+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12021760,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12871,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-94}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15887","reported_seq":16638,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.363843+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:44.363843+0000","last_peered":"2024-09-15T21:25:44.363843+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:44.363843+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12357632,"num_objects":3017,"num_object_clones":0,"num_object_copies":6034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13523,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15436","reported_seq":15569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.330876+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:44.330876+0000","last_peered":"2024-09-15T21:25:44.330876+0000","last_clean":"2024-09-15T21:25:44.330876+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:44.330876+0000","last_undegraded":"2024-09-15T21:25:44.330876+0000","last_fullsized":"2024-09-15T21:25:44.330876+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12156928,"num_objects":2968,"num_object_clones":0,"num_object_copies":5936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15228","reported_seq":22056,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.994843+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:39.994843+0000","last_peered":"2024-09-15T21:25:39.994843+0000","last_clean":"2024-09-15T21:25:39.994843+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:39.994843+0000","last_undegraded":"2024-09-15T21:25:39.994843+0000","last_fullsized":"2024-09-15T21:25:39.994843+0000","mapping_epoch":49,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12890112,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15184","reported_seq":15973,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.361848+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:44.361848+0000","last_peered":"2024-09-15T21:25:44.361848+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:44.361848+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11673600,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15153","reported_seq":17225,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.884858+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:39.884858+0000","last_peered":"2024-09-15T21:25:39.884858+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:39.884858+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12828672,"num_objects":3132,"num_object_clones":0,"num_object_copies":6264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13704,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15370","reported_seq":15786,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:39.863024+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:39.863024+0000","last_peered":"2024-09-15T21:25:39.863024+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:39.863024+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13017088,"num_objects":3178,"num_object_clones":0,"num_object_copies":6356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-28}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15674","reported_seq":16422,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.369773+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:44.369773+0000","last_peered":"2024-09-15T21:25:44.369773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:44.369773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12435456,"num_objects":3036,"num_object_clones":0,"num_object_copies":6072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12981,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-94}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15268","reported_seq":22186,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.877835+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:39.877835+0000","last_peered":"2024-09-15T21:25:39.877835+0000","last_clean":"2024-09-15T21:25:39.877835+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:39.877835+0000","last_undegraded":"2024-09-15T21:25:39.877835+0000","last_fullsized":"2024-09-15T21:25:39.877835+0000","mapping_epoch":51,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15431","reported_seq":16094,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.336406+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:44.336406+0000","last_peered":"2024-09-15T21:25:44.336406+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:44.336406+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12070912,"num_objects":2947,"num_object_clones":0,"num_object_copies":5894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13078,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15523","reported_seq":22621,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:39.958922+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:39.958922+0000","last_peered":"2024-09-15T21:25:39.958922+0000","last_clean":"2024-09-15T21:25:39.958922+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:39.958922+0000","last_undegraded":"2024-09-15T21:25:39.958922+0000","last_fullsized":"2024-09-15T21:25:39.958922+0000","mapping_epoch":47,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12677737,"num_objects":3096,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":201200257,"num_objects":49123,"num_object_clones":0,"num_object_copies":98246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218656,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16583,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570568704,"data_stored":570511854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953574,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503408,"kb_used_data":210720,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93868432,"statfs":{"total":96636764160,"available":96121274368,"internally_reserved":0,"allocated":215777280,"data_stored":215475878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149802,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594180,"kb_used_data":197420,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93777660,"statfs":{"total":96636764160,"available":96028323840,"internally_reserved":0,"allocated":202158080,"data_stored":201860750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512436,"kb_used_data":151716,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93859404,"statfs":{"total":96636764160,"available":96112029696,"internally_reserved":0,"allocated":155357184,"data_stored":155017513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":369356460},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201330688,"data_stored":201327209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":154103808,"data_stored":154057988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215134208,"data_stored":215126657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:44.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:44.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15468", 'reported_seq': 16230, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.362568+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:44.362568+0000', 'last_peered': '2024-09-15T21:25:44.362568+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:44.362568+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12468224, 'num_objects': 3044, 'num_object_clones': 0, 'num_object_copies': 6088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15468, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 13}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:44.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:44.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15446", 'reported_seq': 16273, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.327676+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:44.327676+0000', 'last_peered': '2024-09-15T21:25:44.327676+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:44.327676+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12021760, 'num_objects': 2935, 'num_object_clones': 0, 'num_object_copies': 5870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12871, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -94}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'15887", 'reported_seq': 16638, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.363843+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:44.363843+0000', 'last_peered': '2024-09-15T21:25:44.363843+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:44.363843+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12357632, 'num_objects': 3017, 'num_object_clones': 0, 'num_object_copies': 6034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13523, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'15184", 'reported_seq': 15973, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.361848+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:44.361848+0000', 'last_peered': '2024-09-15T21:25:44.361848+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:44.361848+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11673600, 'num_objects': 2850, 'num_object_clones': 0, 'num_object_copies': 5700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12710, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'15153", 'reported_seq': 17225, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:39.884858+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:39.884858+0000', 'last_peered': '2024-09-15T21:25:39.884858+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:39.884858+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'5100", 'ondisk_log_start': "44'5100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12828672, 'num_objects': 3132, 'num_object_clones': 0, 'num_object_copies': 6264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2294, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13704, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 974, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 838}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15370", 'reported_seq': 15786, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:39.863024+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:39.863024+0000', 'last_peered': '2024-09-15T21:25:39.863024+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:39.863024+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13017088, 'num_objects': 3178, 'num_object_clones': 0, 'num_object_copies': 6356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13769, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -28}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15674", 'reported_seq': 16422, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.369773+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:44.369773+0000', 'last_peered': '2024-09-15T21:25:44.369773+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:44.369773+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12435456, 'num_objects': 3036, 'num_object_clones': 0, 'num_object_copies': 6072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3036, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12981, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -94}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:45.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:45.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'15431", 'reported_seq': 16094, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:44.336406+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:44.336406+0000', 'last_peered': '2024-09-15T21:25:44.336406+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:44.336406+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12070912, 'num_objects': 2947, 'num_object_clones': 0, 'num_object_copies': 5894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13078, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:45.001 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:46.018 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:46.018 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23982/98250 objects degraded (24.409%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":559,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":49125,"data_bytes":201200276,"bytes_used":1648664576,"bytes_avail":288261627904,"bytes_total":289910292480,"degraded_objects":23982,"degraded_total":98250,"degraded_ratio":0.24409160305343514,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":493},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:46.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:46.109 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:46.889 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:46.891 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:46.976 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-15T21:25:46.498343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":196309652,"num_objects":47931,"num_object_clones":0,"num_object_copies":95862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219852,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16717,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1629960,"kb_used_data":547856,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281485560,"statfs":{"total":289910292480,"available":288241213440,"internally_reserved":0,"allocated":561004544,"data_stored":560066141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1108003850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,2,7,16,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24227840,"num_objects":-5915,"num_object_clones":0,"num_object_copies":-11830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006100"},"pg_stats":[{"pgid":"2.f","version":"52'15468","reported_seq":16230,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.362568+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:44.362568+0000","last_peered":"2024-09-15T21:25:44.362568+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:44.362568+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12468224,"num_objects":3044,"num_object_clones":0,"num_object_copies":6088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15468,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":13}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15605","reported_seq":15715,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.812817+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:44.812817+0000","last_peered":"2024-09-15T21:25:44.812817+0000","last_clean":"2024-09-15T21:25:44.812817+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:44.812817+0000","last_undegraded":"2024-09-15T21:25:44.812817+0000","last_fullsized":"2024-09-15T21:25:44.812817+0000","mapping_epoch":36,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12431360,"num_objects":3035,"num_object_clones":0,"num_object_copies":6070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14331,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15721","reported_seq":22651,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:45.065354+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:45.065354+0000","last_peered":"2024-09-15T21:25:45.065354+0000","last_clean":"2024-09-15T21:25:45.065354+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:45.065354+0000","last_undegraded":"2024-09-15T21:25:45.065354+0000","last_fullsized":"2024-09-15T21:25:45.065354+0000","mapping_epoch":45,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12894232,"num_objects":3149,"num_object_clones":0,"num_object_copies":6298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14444,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15629","reported_seq":15712,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.906173+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:44.906173+0000","last_peered":"2024-09-15T21:25:44.906173+0000","last_clean":"2024-09-15T21:25:44.906173+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:44.906173+0000","last_undegraded":"2024-09-15T21:25:44.906173+0000","last_fullsized":"2024-09-15T21:25:44.906173+0000","mapping_epoch":36,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12046336,"num_objects":2941,"num_object_clones":0,"num_object_copies":5882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15629,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15673","reported_seq":15805,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.325697+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:44.325697+0000","last_peered":"2024-09-15T21:25:44.325697+0000","last_clean":"2024-09-15T21:25:44.325697+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:44.325697+0000","last_undegraded":"2024-09-15T21:25:44.325697+0000","last_fullsized":"2024-09-15T21:25:44.325697+0000","mapping_epoch":35,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13531,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15446","reported_seq":16273,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.327676+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:44.327676+0000","last_peered":"2024-09-15T21:25:44.327676+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:44.327676+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12021760,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12871,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-94}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'15887","reported_seq":16638,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.363843+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:44.363843+0000","last_peered":"2024-09-15T21:25:44.363843+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:44.363843+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12357632,"num_objects":3017,"num_object_clones":0,"num_object_copies":6034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13523,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15436","reported_seq":15569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:44.330876+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:44.330876+0000","last_peered":"2024-09-15T21:25:44.330876+0000","last_clean":"2024-09-15T21:25:44.330876+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:44.330876+0000","last_undegraded":"2024-09-15T21:25:44.330876+0000","last_fullsized":"2024-09-15T21:25:44.330876+0000","mapping_epoch":35,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12156928,"num_objects":2968,"num_object_clones":0,"num_object_copies":5936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15385","reported_seq":22213,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:45.057753+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:45.057753+0000","last_peered":"2024-09-15T21:25:45.057753+0000","last_clean":"2024-09-15T21:25:45.057753+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:45.057753+0000","last_undegraded":"2024-09-15T21:25:45.057753+0000","last_fullsized":"2024-09-15T21:25:45.057753+0000","mapping_epoch":49,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12247040,"num_objects":2990,"num_object_clones":0,"num_object_copies":5980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13971,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15184","reported_seq":15973,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.361848+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:44.361848+0000","last_peered":"2024-09-15T21:25:44.361848+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:44.361848+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5100","ondisk_log_start":"44'5100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11673600,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12710,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15300","reported_seq":17638,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.861184+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:44.861184+0000","last_peered":"2024-09-15T21:25:44.861184+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:44.861184+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5200","ondisk_log_start":"44'5200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12226560,"num_objects":2985,"num_object_clones":0,"num_object_copies":5970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13851,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":824}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15507","reported_seq":15924,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.821329+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:44.821329+0000","last_peered":"2024-09-15T21:25:44.821329+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:44.821329+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12455936,"num_objects":3041,"num_object_clones":0,"num_object_copies":6082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13906,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-165}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15674","reported_seq":16422,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.369773+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:44.369773+0000","last_peered":"2024-09-15T21:25:44.369773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:44.369773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12435456,"num_objects":3036,"num_object_clones":0,"num_object_copies":6072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12981,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-94}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15413","reported_seq":22331,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:45.064510+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:45.064510+0000","last_peered":"2024-09-15T21:25:45.064510+0000","last_clean":"2024-09-15T21:25:45.064510+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:45.064510+0000","last_undegraded":"2024-09-15T21:25:45.064510+0000","last_fullsized":"2024-09-15T21:25:45.064510+0000","mapping_epoch":51,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12283904,"num_objects":2999,"num_object_clones":0,"num_object_copies":5998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12998,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15431","reported_seq":16094,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:44.336406+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:44.336406+0000","last_peered":"2024-09-15T21:25:44.336406+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:44.336406+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12070912,"num_objects":2947,"num_object_clones":0,"num_object_copies":5894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13078,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15677","reported_seq":22775,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:45.020373+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:45.020373+0000","last_peered":"2024-09-15T21:25:45.020373+0000","last_clean":"2024-09-15T21:25:45.020373+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:45.020373+0000","last_undegraded":"2024-09-15T21:25:45.020373+0000","last_fullsized":"2024-09-15T21:25:45.020373+0000","mapping_epoch":47,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12046953,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13337,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":196309633,"num_objects":47929,"num_object_clones":0,"num_object_copies":95858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219850,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16716,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":558395392,"data_stored":558338542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953575,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":205924,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":210866176,"data_stored":210564774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149802,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594180,"kb_used_data":197420,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93777660,"statfs":{"total":96636764160,"available":96028323840,"internally_reserved":0,"allocated":202158080,"data_stored":201860750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626521,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549648,"kb_used_data":144512,"kb_used_omap":4,"kb_used_meta":405115,"kb_avail":93822192,"statfs":{"total":96636764160,"available":96073924608,"internally_reserved":0,"allocated":147980288,"data_stored":147640617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":414838444},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,4,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":201330688,"data_stored":201327209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146780160,"data_stored":146734340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210284544,"data_stored":210276993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:49.978 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:50.742 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:50.778 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-15T21:25:50.500073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186401428,"num_objects":45512,"num_object_clones":0,"num_object_copies":91024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222271,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16817,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1606748,"kb_used_data":528808,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281508772,"statfs":{"total":289910292480,"available":288264982528,"internally_reserved":0,"allocated":541499392,"data_stored":540560989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1103744010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,8,12,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29532160,"num_objects":-7210,"num_object_clones":0,"num_object_copies":-14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7210,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006922"},"pg_stats":[{"pgid":"2.f","version":"52'15625","reported_seq":16387,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.412281+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:49.412281+0000","last_peered":"2024-09-15T21:25:49.412281+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:49.412281+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11825152,"num_objects":2887,"num_object_clones":0,"num_object_copies":5774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15625,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15775","reported_seq":15885,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.859785+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:49.859785+0000","last_peered":"2024-09-15T21:25:49.859785+0000","last_clean":"2024-09-15T21:25:49.859785+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:49.859785+0000","last_undegraded":"2024-09-15T21:25:49.859785+0000","last_fullsized":"2024-09-15T21:25:49.859785+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11735040,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14501,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15861","reported_seq":22791,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.000719+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:50.000719+0000","last_peered":"2024-09-15T21:25:50.000719+0000","last_clean":"2024-09-15T21:25:50.000719+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:50.000719+0000","last_undegraded":"2024-09-15T21:25:50.000719+0000","last_fullsized":"2024-09-15T21:25:50.000719+0000","mapping_epoch":45,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320792,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14584,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15783","reported_seq":15866,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.898326+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:49.898326+0000","last_peered":"2024-09-15T21:25:49.898326+0000","last_clean":"2024-09-15T21:25:49.898326+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:49.898326+0000","last_undegraded":"2024-09-15T21:25:49.898326+0000","last_fullsized":"2024-09-15T21:25:49.898326+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11415552,"num_objects":2787,"num_object_clones":0,"num_object_copies":5574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15783,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15823","reported_seq":15955,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.352198+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:49.352198+0000","last_peered":"2024-09-15T21:25:49.352198+0000","last_clean":"2024-09-15T21:25:49.352198+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:49.352198+0000","last_undegraded":"2024-09-15T21:25:49.352198+0000","last_fullsized":"2024-09-15T21:25:49.352198+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11878400,"num_objects":2900,"num_object_clones":0,"num_object_copies":5800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13681,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15578","reported_seq":16401,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.457281+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:49.457281+0000","last_peered":"2024-09-15T21:25:49.457281+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:49.457281+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16041","reported_seq":16791,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.373314+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:49.373314+0000","last_peered":"2024-09-15T21:25:49.373314+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:49.373314+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13677,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15566","reported_seq":15699,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.418915+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:49.418915+0000","last_peered":"2024-09-15T21:25:49.418915+0000","last_clean":"2024-09-15T21:25:49.418915+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:49.418915+0000","last_undegraded":"2024-09-15T21:25:49.418915+0000","last_fullsized":"2024-09-15T21:25:49.418915+0000","mapping_epoch":35,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11624448,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15527","reported_seq":22355,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.092786+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:50.092786+0000","last_peered":"2024-09-15T21:25:50.092786+0000","last_clean":"2024-09-15T21:25:50.092786+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:50.092786+0000","last_undegraded":"2024-09-15T21:25:50.092786+0000","last_fullsized":"2024-09-15T21:25:50.092786+0000","mapping_epoch":49,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14113,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15324","reported_seq":16111,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.368441+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:49.368441+0000","last_peered":"2024-09-15T21:25:49.368441+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:49.368441+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15449","reported_seq":17987,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.844310+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:49.844310+0000","last_peered":"2024-09-15T21:25:49.844310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:49.844310+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11616256,"num_objects":2836,"num_object_clones":0,"num_object_copies":5672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14000,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15686","reported_seq":16103,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.883998+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:49.883998+0000","last_peered":"2024-09-15T21:25:49.883998+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:49.883998+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15833","reported_seq":16581,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.396310+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:49.396310+0000","last_peered":"2024-09-15T21:25:49.396310+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:49.396310+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784192,"num_objects":2877,"num_object_clones":0,"num_object_copies":5754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13140,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15564","reported_seq":22481,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.120799+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:50.120799+0000","last_peered":"2024-09-15T21:25:50.120799+0000","last_clean":"2024-09-15T21:25:50.120799+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:50.120799+0000","last_undegraded":"2024-09-15T21:25:50.120799+0000","last_fullsized":"2024-09-15T21:25:50.120799+0000","mapping_epoch":51,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13149,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15589","reported_seq":16252,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.403787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:49.403787+0000","last_peered":"2024-09-15T21:25:49.403787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:49.403787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11423744,"num_objects":2789,"num_object_clones":0,"num_object_copies":5578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13236,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15831","reported_seq":22929,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.111404+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:50.111404+0000","last_peered":"2024-09-15T21:25:50.111404+0000","last_clean":"2024-09-15T21:25:50.111404+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:50.111404+0000","last_undegraded":"2024-09-15T21:25:50.111404+0000","last_fullsized":"2024-09-15T21:25:50.111404+0000","mapping_epoch":47,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11416169,"num_objects":2788,"num_object_clones":0,"num_object_copies":5576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186401409,"num_objects":45510,"num_object_clones":0,"num_object_copies":91020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222269,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16816,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":538505216,"data_stored":538448366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953576,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485404,"kb_used_data":201036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93886436,"statfs":{"total":96636764160,"available":96139710464,"internally_reserved":0,"allocated":205860864,"data_stored":205559462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591360,"kb_used_data":190448,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93780480,"statfs":{"total":96636764160,"available":96031211520,"internally_reserved":0,"allocated":195018752,"data_stored":194721422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":410513073},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529984,"kb_used_data":137324,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93841856,"statfs":{"total":96636764160,"available":96094060544,"internally_reserved":0,"allocated":140619776,"data_stored":140280105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":402058924},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194027520,"data_stored":194024041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139206656,"data_stored":139160836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205271040,"data_stored":205263489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:50.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:51.587 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:51.645 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-15T21:25:50.500073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186401428,"num_objects":45512,"num_object_clones":0,"num_object_copies":91024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222271,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16817,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1606748,"kb_used_data":528808,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281508772,"statfs":{"total":289910292480,"available":288264982528,"internally_reserved":0,"allocated":541499392,"data_stored":540560989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1103744010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,8,12,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29532160,"num_objects":-7210,"num_object_clones":0,"num_object_copies":-14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7210,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":335,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006922"},"pg_stats":[{"pgid":"2.f","version":"52'15625","reported_seq":16387,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.412281+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:49.412281+0000","last_peered":"2024-09-15T21:25:49.412281+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:49.412281+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11825152,"num_objects":2887,"num_object_clones":0,"num_object_copies":5774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15625,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15775","reported_seq":15885,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.859785+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:49.859785+0000","last_peered":"2024-09-15T21:25:49.859785+0000","last_clean":"2024-09-15T21:25:49.859785+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:49.859785+0000","last_undegraded":"2024-09-15T21:25:49.859785+0000","last_fullsized":"2024-09-15T21:25:49.859785+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11735040,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14501,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15861","reported_seq":22791,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.000719+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:50.000719+0000","last_peered":"2024-09-15T21:25:50.000719+0000","last_clean":"2024-09-15T21:25:50.000719+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:50.000719+0000","last_undegraded":"2024-09-15T21:25:50.000719+0000","last_fullsized":"2024-09-15T21:25:50.000719+0000","mapping_epoch":45,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320792,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14584,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15783","reported_seq":15866,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.898326+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:49.898326+0000","last_peered":"2024-09-15T21:25:49.898326+0000","last_clean":"2024-09-15T21:25:49.898326+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:49.898326+0000","last_undegraded":"2024-09-15T21:25:49.898326+0000","last_fullsized":"2024-09-15T21:25:49.898326+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11415552,"num_objects":2787,"num_object_clones":0,"num_object_copies":5574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15783,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15823","reported_seq":15955,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.352198+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:49.352198+0000","last_peered":"2024-09-15T21:25:49.352198+0000","last_clean":"2024-09-15T21:25:49.352198+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:49.352198+0000","last_undegraded":"2024-09-15T21:25:49.352198+0000","last_fullsized":"2024-09-15T21:25:49.352198+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11878400,"num_objects":2900,"num_object_clones":0,"num_object_copies":5800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13681,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15578","reported_seq":16401,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.457281+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:49.457281+0000","last_peered":"2024-09-15T21:25:49.457281+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:49.457281+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16041","reported_seq":16791,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.373314+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:49.373314+0000","last_peered":"2024-09-15T21:25:49.373314+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:49.373314+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13677,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15566","reported_seq":15699,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.418915+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:49.418915+0000","last_peered":"2024-09-15T21:25:49.418915+0000","last_clean":"2024-09-15T21:25:49.418915+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:49.418915+0000","last_undegraded":"2024-09-15T21:25:49.418915+0000","last_fullsized":"2024-09-15T21:25:49.418915+0000","mapping_epoch":35,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11624448,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15527","reported_seq":22355,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.092786+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:50.092786+0000","last_peered":"2024-09-15T21:25:50.092786+0000","last_clean":"2024-09-15T21:25:50.092786+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:50.092786+0000","last_undegraded":"2024-09-15T21:25:50.092786+0000","last_fullsized":"2024-09-15T21:25:50.092786+0000","mapping_epoch":49,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14113,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15324","reported_seq":16111,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.368441+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:49.368441+0000","last_peered":"2024-09-15T21:25:49.368441+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:49.368441+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15449","reported_seq":17987,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.844310+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:49.844310+0000","last_peered":"2024-09-15T21:25:49.844310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:49.844310+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11616256,"num_objects":2836,"num_object_clones":0,"num_object_copies":5672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14000,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15686","reported_seq":16103,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.883998+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:49.883998+0000","last_peered":"2024-09-15T21:25:49.883998+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:49.883998+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15833","reported_seq":16581,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.396310+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:49.396310+0000","last_peered":"2024-09-15T21:25:49.396310+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:49.396310+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784192,"num_objects":2877,"num_object_clones":0,"num_object_copies":5754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13140,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15564","reported_seq":22481,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.120799+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:50.120799+0000","last_peered":"2024-09-15T21:25:50.120799+0000","last_clean":"2024-09-15T21:25:50.120799+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:50.120799+0000","last_undegraded":"2024-09-15T21:25:50.120799+0000","last_fullsized":"2024-09-15T21:25:50.120799+0000","mapping_epoch":51,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13149,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15589","reported_seq":16252,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.403787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:49.403787+0000","last_peered":"2024-09-15T21:25:49.403787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:49.403787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11423744,"num_objects":2789,"num_object_clones":0,"num_object_copies":5578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13236,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15831","reported_seq":22929,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.111404+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:50.111404+0000","last_peered":"2024-09-15T21:25:50.111404+0000","last_clean":"2024-09-15T21:25:50.111404+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:50.111404+0000","last_undegraded":"2024-09-15T21:25:50.111404+0000","last_fullsized":"2024-09-15T21:25:50.111404+0000","mapping_epoch":47,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11416169,"num_objects":2788,"num_object_clones":0,"num_object_copies":5576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186401409,"num_objects":45510,"num_object_clones":0,"num_object_copies":91020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222269,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16816,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":538505216,"data_stored":538448366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953576,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485404,"kb_used_data":201036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93886436,"statfs":{"total":96636764160,"available":96139710464,"internally_reserved":0,"allocated":205860864,"data_stored":205559462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591360,"kb_used_data":190448,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93780480,"statfs":{"total":96636764160,"available":96031211520,"internally_reserved":0,"allocated":195018752,"data_stored":194721422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":410513073},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529984,"kb_used_data":137324,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93841856,"statfs":{"total":96636764160,"available":96094060544,"internally_reserved":0,"allocated":140619776,"data_stored":140280105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":402058924},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194027520,"data_stored":194024041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139206656,"data_stored":139160836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205271040,"data_stored":205263489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:51.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:51.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15625", 'reported_seq': 16387, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.412281+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:49.412281+0000', 'last_peered': '2024-09-15T21:25:49.412281+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:49.412281+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11825152, 'num_objects': 2887, 'num_object_clones': 0, 'num_object_copies': 5774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15625, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -144}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:51.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15578", 'reported_seq': 16401, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.457281+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:49.457281+0000', 'last_peered': '2024-09-15T21:25:49.457281+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:49.457281+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11481088, 'num_objects': 2803, 'num_object_clones': 0, 'num_object_copies': 5606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13003, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16041", 'reported_seq': 16791, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.373314+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:49.373314+0000', 'last_peered': '2024-09-15T21:25:49.373314+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:49.373314+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11726848, 'num_objects': 2863, 'num_object_clones': 0, 'num_object_copies': 5726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13677, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'15324", 'reported_seq': 16111, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.368441+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:49.368441+0000', 'last_peered': '2024-09-15T21:25:49.368441+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:49.368441+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'5300", 'ondisk_log_start': "44'5300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11100160, 'num_objects': 2710, 'num_object_clones': 0, 'num_object_copies': 5420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12850, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -305}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'15449", 'reported_seq': 17987, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.844310+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:49.844310+0000', 'last_peered': '2024-09-15T21:25:49.844310+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:49.844310+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11616256, 'num_objects': 2836, 'num_object_clones': 0, 'num_object_copies': 5672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2836, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14000, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1207, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 775}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15686", 'reported_seq': 16103, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.883998+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:49.883998+0000', 'last_peered': '2024-09-15T21:25:49.883998+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:49.883998+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11722752, 'num_objects': 2862, 'num_object_clones': 0, 'num_object_copies': 5724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14085, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -344}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15833", 'reported_seq': 16581, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.396310+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:49.396310+0000', 'last_peered': '2024-09-15T21:25:49.396310+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:49.396310+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11784192, 'num_objects': 2877, 'num_object_clones': 0, 'num_object_copies': 5754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13140, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:51.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'15589", 'reported_seq': 16252, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:49.403787+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:49.403787+0000', 'last_peered': '2024-09-15T21:25:49.403787+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:49.403787+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11423744, 'num_objects': 2789, 'num_object_clones': 0, 'num_object_copies': 5578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2789, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13236, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:51.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:52.469 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:52.469 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23749/91024 objects degraded (26.091%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":45512,"data_bytes":186401428,"bytes_used":1645309952,"bytes_avail":288264982528,"bytes_total":289910292480,"degraded_objects":23749,"degraded_total":91024,"degraded_ratio":0.26090921075760237,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":335,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":600},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:52.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:52.509 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:53.201 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:53.254 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:53.254 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-15T21:25:52.504443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186401428,"num_objects":45512,"num_object_clones":0,"num_object_copies":91024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45512,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222271,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16817,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1606748,"kb_used_data":528808,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281508772,"statfs":{"total":289910292480,"available":288264982528,"internally_reserved":0,"allocated":541499392,"data_stored":540560989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1103744010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,8,12,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19722240,"num_objects":-4815,"num_object_clones":0,"num_object_copies":-9630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4815,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007399"},"pg_stats":[{"pgid":"2.f","version":"52'15625","reported_seq":16387,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.412281+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:49.412281+0000","last_peered":"2024-09-15T21:25:49.412281+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:49.412281+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11825152,"num_objects":2887,"num_object_clones":0,"num_object_copies":5774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15625,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15775","reported_seq":15885,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.859785+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:49.859785+0000","last_peered":"2024-09-15T21:25:49.859785+0000","last_clean":"2024-09-15T21:25:49.859785+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:49.859785+0000","last_undegraded":"2024-09-15T21:25:49.859785+0000","last_fullsized":"2024-09-15T21:25:49.859785+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11735040,"num_objects":2865,"num_object_clones":0,"num_object_copies":5730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14501,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'15861","reported_seq":22791,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.000719+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:50.000719+0000","last_peered":"2024-09-15T21:25:50.000719+0000","last_clean":"2024-09-15T21:25:50.000719+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:50.000719+0000","last_undegraded":"2024-09-15T21:25:50.000719+0000","last_fullsized":"2024-09-15T21:25:50.000719+0000","mapping_epoch":45,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320792,"num_objects":3009,"num_object_clones":0,"num_object_copies":6018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14584,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15783","reported_seq":15866,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.898326+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:49.898326+0000","last_peered":"2024-09-15T21:25:49.898326+0000","last_clean":"2024-09-15T21:25:49.898326+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:49.898326+0000","last_undegraded":"2024-09-15T21:25:49.898326+0000","last_fullsized":"2024-09-15T21:25:49.898326+0000","mapping_epoch":36,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11415552,"num_objects":2787,"num_object_clones":0,"num_object_copies":5574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15783,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15823","reported_seq":15955,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.352198+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:49.352198+0000","last_peered":"2024-09-15T21:25:49.352198+0000","last_clean":"2024-09-15T21:25:49.352198+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:49.352198+0000","last_undegraded":"2024-09-15T21:25:49.352198+0000","last_fullsized":"2024-09-15T21:25:49.352198+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11878400,"num_objects":2900,"num_object_clones":0,"num_object_copies":5800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13681,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15578","reported_seq":16401,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.457281+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:49.457281+0000","last_peered":"2024-09-15T21:25:49.457281+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:49.457281+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16041","reported_seq":16791,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.373314+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:49.373314+0000","last_peered":"2024-09-15T21:25:49.373314+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:49.373314+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11726848,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13677,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15566","reported_seq":15699,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:49.418915+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:49.418915+0000","last_peered":"2024-09-15T21:25:49.418915+0000","last_clean":"2024-09-15T21:25:49.418915+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:49.418915+0000","last_undegraded":"2024-09-15T21:25:49.418915+0000","last_fullsized":"2024-09-15T21:25:49.418915+0000","mapping_epoch":35,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11624448,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15527","reported_seq":22355,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.092786+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:50.092786+0000","last_peered":"2024-09-15T21:25:50.092786+0000","last_clean":"2024-09-15T21:25:50.092786+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:50.092786+0000","last_undegraded":"2024-09-15T21:25:50.092786+0000","last_fullsized":"2024-09-15T21:25:50.092786+0000","mapping_epoch":49,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14113,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15324","reported_seq":16111,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.368441+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:49.368441+0000","last_peered":"2024-09-15T21:25:49.368441+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:49.368441+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5300","ondisk_log_start":"44'5300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11100160,"num_objects":2710,"num_object_clones":0,"num_object_copies":5420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12850,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15449","reported_seq":17987,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.844310+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:49.844310+0000","last_peered":"2024-09-15T21:25:49.844310+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:49.844310+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11616256,"num_objects":2836,"num_object_clones":0,"num_object_copies":5672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14000,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":775}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15686","reported_seq":16103,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.883998+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:49.883998+0000","last_peered":"2024-09-15T21:25:49.883998+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:49.883998+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14085,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-344}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15833","reported_seq":16581,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.396310+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:49.396310+0000","last_peered":"2024-09-15T21:25:49.396310+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:49.396310+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784192,"num_objects":2877,"num_object_clones":0,"num_object_copies":5754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13140,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15564","reported_seq":22481,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.120799+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:50.120799+0000","last_peered":"2024-09-15T21:25:50.120799+0000","last_clean":"2024-09-15T21:25:50.120799+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:50.120799+0000","last_undegraded":"2024-09-15T21:25:50.120799+0000","last_fullsized":"2024-09-15T21:25:50.120799+0000","mapping_epoch":51,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":2848,"num_object_clones":0,"num_object_copies":5696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13149,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15589","reported_seq":16252,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:49.403787+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:49.403787+0000","last_peered":"2024-09-15T21:25:49.403787+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:49.403787+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11423744,"num_objects":2789,"num_object_clones":0,"num_object_copies":5578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13236,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15831","reported_seq":22929,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:50.111404+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:50.111404+0000","last_peered":"2024-09-15T21:25:50.111404+0000","last_clean":"2024-09-15T21:25:50.111404+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:50.111404+0000","last_undegraded":"2024-09-15T21:25:50.111404+0000","last_fullsized":"2024-09-15T21:25:50.111404+0000","mapping_epoch":47,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11416169,"num_objects":2788,"num_object_clones":0,"num_object_copies":5576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186401409,"num_objects":45510,"num_object_clones":0,"num_object_copies":91020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23749,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222269,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16816,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":538505216,"data_stored":538448366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953576,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485404,"kb_used_data":201036,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93886436,"statfs":{"total":96636764160,"available":96139710464,"internally_reserved":0,"allocated":205860864,"data_stored":205559462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149803,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591360,"kb_used_data":190448,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93780480,"statfs":{"total":96636764160,"available":96031211520,"internally_reserved":0,"allocated":195018752,"data_stored":194721422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":410513073},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529984,"kb_used_data":137324,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93841856,"statfs":{"total":96636764160,"available":96094060544,"internally_reserved":0,"allocated":140619776,"data_stored":140280105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":402058924},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194027520,"data_stored":194024041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139206656,"data_stored":139160836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205271040,"data_stored":205263489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:56.256 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:57.021 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:57.023 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:57.048 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-15T21:25:56.508415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":176620180,"num_objects":43124,"num_object_clones":0,"num_object_copies":86248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224659,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16940,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609852,"kb_used_data":509688,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281505668,"statfs":{"total":289910292480,"available":288261804032,"internally_reserved":0,"allocated":521920512,"data_stored":520982109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,6,3,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24580096,"num_objects":-6001,"num_object_clones":0,"num_object_copies":-12002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6001,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010439"},"pg_stats":[{"pgid":"2.f","version":"52'15805","reported_seq":16567,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.455786+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:54.455786+0000","last_peered":"2024-09-15T21:25:54.455786+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:54.455786+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087872,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15805,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15936","reported_seq":16046,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.885493+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:54.885493+0000","last_peered":"2024-09-15T21:25:54.885493+0000","last_clean":"2024-09-15T21:25:54.885493+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:54.885493+0000","last_undegraded":"2024-09-15T21:25:54.885493+0000","last_fullsized":"2024-09-15T21:25:54.885493+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16009","reported_seq":22939,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.099926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:55.099926+0000","last_peered":"2024-09-15T21:25:55.099926+0000","last_clean":"2024-09-15T21:25:55.099926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:55.099926+0000","last_undegraded":"2024-09-15T21:25:55.099926+0000","last_fullsized":"2024-09-15T21:25:55.099926+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11714584,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14732,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15934","reported_seq":16017,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.853711+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:54.853711+0000","last_peered":"2024-09-15T21:25:54.853711+0000","last_clean":"2024-09-15T21:25:54.853711+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:54.853711+0000","last_undegraded":"2024-09-15T21:25:54.853711+0000","last_fullsized":"2024-09-15T21:25:54.853711+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15991","reported_seq":16123,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.290371+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:54.290371+0000","last_peered":"2024-09-15T21:25:54.290371+0000","last_clean":"2024-09-15T21:25:54.290371+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:54.290371+0000","last_undegraded":"2024-09-15T21:25:54.290371+0000","last_fullsized":"2024-09-15T21:25:54.290371+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15712","reported_seq":16539,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.417234+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:54.417234+0000","last_peered":"2024-09-15T21:25:54.417234+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:54.417234+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16191","reported_seq":16937,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.476152+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:54.476152+0000","last_peered":"2024-09-15T21:25:54.476152+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:54.476152+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15706","reported_seq":15839,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.398363+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:54.398363+0000","last_peered":"2024-09-15T21:25:54.398363+0000","last_clean":"2024-09-15T21:25:54.398363+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:54.398363+0000","last_undegraded":"2024-09-15T21:25:54.398363+0000","last_fullsized":"2024-09-15T21:25:54.398363+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11051008,"num_objects":2698,"num_object_clones":0,"num_object_copies":5396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15684","reported_seq":22512,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.134391+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:55.134391+0000","last_peered":"2024-09-15T21:25:55.134391+0000","last_clean":"2024-09-15T21:25:55.134391+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:55.134391+0000","last_undegraded":"2024-09-15T21:25:55.134391+0000","last_fullsized":"2024-09-15T21:25:55.134391+0000","mapping_epoch":49,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15487","reported_seq":16276,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.450551+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:54.450551+0000","last_peered":"2024-09-15T21:25:54.450551+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:54.450551+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10432512,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13013,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15584","reported_seq":18368,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.913303+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:54.913303+0000","last_peered":"2024-09-15T21:25:54.913303+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:54.913303+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15821","reported_seq":16238,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.891327+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:54.891327+0000","last_peered":"2024-09-15T21:25:54.891327+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:54.891327+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11169792,"num_objects":2727,"num_object_clones":0,"num_object_copies":5454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14220,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15987","reported_seq":16735,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.456419+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:54.456419+0000","last_peered":"2024-09-15T21:25:54.456419+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:54.456419+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11153408,"num_objects":2723,"num_object_clones":0,"num_object_copies":5446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15694","reported_seq":22612,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.115712+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:55.115712+0000","last_peered":"2024-09-15T21:25:55.115712+0000","last_clean":"2024-09-15T21:25:55.115712+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:55.115712+0000","last_undegraded":"2024-09-15T21:25:55.115712+0000","last_fullsized":"2024-09-15T21:25:55.115712+0000","mapping_epoch":51,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11132928,"num_objects":2718,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13279,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15729","reported_seq":16392,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.460744+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:54.460744+0000","last_peered":"2024-09-15T21:25:54.460744+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:54.460744+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10850304,"num_objects":2649,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15973","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.062676+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:55.062676+0000","last_peered":"2024-09-15T21:25:55.062676+0000","last_clean":"2024-09-15T21:25:55.062676+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:55.062676+0000","last_undegraded":"2024-09-15T21:25:55.062676+0000","last_fullsized":"2024-09-15T21:25:55.062676+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834537,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":176620161,"num_objects":43122,"num_object_clones":0,"num_object_copies":86244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224657,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16939,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":518955008,"data_stored":518898158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484820,"kb_used_data":196288,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93887020,"statfs":{"total":96636764160,"available":96140308480,"internally_reserved":0,"allocated":200998912,"data_stored":200697510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575756,"kb_used_data":183152,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93796084,"statfs":{"total":96636764160,"available":96047190016,"internally_reserved":0,"allocated":187547648,"data_stored":187250318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549276,"kb_used_data":130248,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93822564,"statfs":{"total":96636764160,"available":96074305536,"internally_reserved":0,"allocated":133373952,"data_stored":133034281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,2,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186580992,"data_stored":186577513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131973120,"data_stored":131927300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200400896,"data_stored":200393345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:57.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:57.788 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:57.867 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-15T21:25:56.508415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":176620180,"num_objects":43124,"num_object_clones":0,"num_object_copies":86248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224659,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16940,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609852,"kb_used_data":509688,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281505668,"statfs":{"total":289910292480,"available":288261804032,"internally_reserved":0,"allocated":521920512,"data_stored":520982109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,6,3,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24580096,"num_objects":-6001,"num_object_clones":0,"num_object_copies":-12002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-356,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6001,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010439"},"pg_stats":[{"pgid":"2.f","version":"52'15805","reported_seq":16567,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.455786+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:54.455786+0000","last_peered":"2024-09-15T21:25:54.455786+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:54.455786+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087872,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15805,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15936","reported_seq":16046,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.885493+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:54.885493+0000","last_peered":"2024-09-15T21:25:54.885493+0000","last_clean":"2024-09-15T21:25:54.885493+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:54.885493+0000","last_undegraded":"2024-09-15T21:25:54.885493+0000","last_fullsized":"2024-09-15T21:25:54.885493+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16009","reported_seq":22939,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.099926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:55.099926+0000","last_peered":"2024-09-15T21:25:55.099926+0000","last_clean":"2024-09-15T21:25:55.099926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:55.099926+0000","last_undegraded":"2024-09-15T21:25:55.099926+0000","last_fullsized":"2024-09-15T21:25:55.099926+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11714584,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14732,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15934","reported_seq":16017,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.853711+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:54.853711+0000","last_peered":"2024-09-15T21:25:54.853711+0000","last_clean":"2024-09-15T21:25:54.853711+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:54.853711+0000","last_undegraded":"2024-09-15T21:25:54.853711+0000","last_fullsized":"2024-09-15T21:25:54.853711+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15991","reported_seq":16123,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.290371+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:54.290371+0000","last_peered":"2024-09-15T21:25:54.290371+0000","last_clean":"2024-09-15T21:25:54.290371+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:54.290371+0000","last_undegraded":"2024-09-15T21:25:54.290371+0000","last_fullsized":"2024-09-15T21:25:54.290371+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15712","reported_seq":16539,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.417234+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:54.417234+0000","last_peered":"2024-09-15T21:25:54.417234+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:54.417234+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16191","reported_seq":16937,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.476152+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:54.476152+0000","last_peered":"2024-09-15T21:25:54.476152+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:54.476152+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15706","reported_seq":15839,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.398363+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:54.398363+0000","last_peered":"2024-09-15T21:25:54.398363+0000","last_clean":"2024-09-15T21:25:54.398363+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:54.398363+0000","last_undegraded":"2024-09-15T21:25:54.398363+0000","last_fullsized":"2024-09-15T21:25:54.398363+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11051008,"num_objects":2698,"num_object_clones":0,"num_object_copies":5396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15684","reported_seq":22512,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.134391+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:55.134391+0000","last_peered":"2024-09-15T21:25:55.134391+0000","last_clean":"2024-09-15T21:25:55.134391+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:55.134391+0000","last_undegraded":"2024-09-15T21:25:55.134391+0000","last_fullsized":"2024-09-15T21:25:55.134391+0000","mapping_epoch":49,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15487","reported_seq":16276,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.450551+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:54.450551+0000","last_peered":"2024-09-15T21:25:54.450551+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:54.450551+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10432512,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13013,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15584","reported_seq":18368,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.913303+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:54.913303+0000","last_peered":"2024-09-15T21:25:54.913303+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:54.913303+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15821","reported_seq":16238,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.891327+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:54.891327+0000","last_peered":"2024-09-15T21:25:54.891327+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:54.891327+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11169792,"num_objects":2727,"num_object_clones":0,"num_object_copies":5454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14220,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15987","reported_seq":16735,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.456419+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:54.456419+0000","last_peered":"2024-09-15T21:25:54.456419+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:54.456419+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11153408,"num_objects":2723,"num_object_clones":0,"num_object_copies":5446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15694","reported_seq":22612,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.115712+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:55.115712+0000","last_peered":"2024-09-15T21:25:55.115712+0000","last_clean":"2024-09-15T21:25:55.115712+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:55.115712+0000","last_undegraded":"2024-09-15T21:25:55.115712+0000","last_fullsized":"2024-09-15T21:25:55.115712+0000","mapping_epoch":51,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11132928,"num_objects":2718,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13279,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15729","reported_seq":16392,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.460744+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:54.460744+0000","last_peered":"2024-09-15T21:25:54.460744+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:54.460744+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10850304,"num_objects":2649,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15973","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.062676+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:55.062676+0000","last_peered":"2024-09-15T21:25:55.062676+0000","last_clean":"2024-09-15T21:25:55.062676+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:55.062676+0000","last_undegraded":"2024-09-15T21:25:55.062676+0000","last_fullsized":"2024-09-15T21:25:55.062676+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834537,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":176620161,"num_objects":43122,"num_object_clones":0,"num_object_copies":86244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224657,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16939,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":518955008,"data_stored":518898158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484820,"kb_used_data":196288,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93887020,"statfs":{"total":96636764160,"available":96140308480,"internally_reserved":0,"allocated":200998912,"data_stored":200697510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575756,"kb_used_data":183152,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93796084,"statfs":{"total":96636764160,"available":96047190016,"internally_reserved":0,"allocated":187547648,"data_stored":187250318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549276,"kb_used_data":130248,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93822564,"statfs":{"total":96636764160,"available":96074305536,"internally_reserved":0,"allocated":133373952,"data_stored":133034281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,2,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186580992,"data_stored":186577513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131973120,"data_stored":131927300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200400896,"data_stored":200393345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:57.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:57.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15805", 'reported_seq': 16567, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.455786+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:54.455786+0000', 'last_peered': '2024-09-15T21:25:54.455786+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:54.455786+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11087872, 'num_objects': 2707, 'num_object_clones': 0, 'num_object_copies': 5414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15805, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -324}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:57.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15712", 'reported_seq': 16539, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.417234+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:54.417234+0000', 'last_peered': '2024-09-15T21:25:54.417234+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:54.417234+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10932224, 'num_objects': 2669, 'num_object_clones': 0, 'num_object_copies': 5338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2669, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13137, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -360}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16191", 'reported_seq': 16937, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.476152+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:54.476152+0000', 'last_peered': '2024-09-15T21:25:54.476152+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:54.476152+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11112448, 'num_objects': 2713, 'num_object_clones': 0, 'num_object_copies': 5426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13827, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'15487", 'reported_seq': 16276, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.450551+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:54.450551+0000', 'last_peered': '2024-09-15T21:25:54.450551+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:54.450551+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'5400", 'ondisk_log_start': "44'5400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10432512, 'num_objects': 2547, 'num_object_clones': 0, 'num_object_copies': 5094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13013, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -468}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'15584", 'reported_seq': 18368, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.913303+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:54.913303+0000', 'last_peered': '2024-09-15T21:25:54.913303+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:54.913303+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'5500", 'ondisk_log_start': "44'5500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11063296, 'num_objects': 2701, 'num_object_clones': 0, 'num_object_copies': 5402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1938, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2701, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14135, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1330, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 763}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15821", 'reported_seq': 16238, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.891327+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:54.891327+0000', 'last_peered': '2024-09-15T21:25:54.891327+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:54.891327+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11169792, 'num_objects': 2727, 'num_object_clones': 0, 'num_object_copies': 5454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14220, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -479}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:57.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:57.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'15987", 'reported_seq': 16735, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.456419+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:54.456419+0000', 'last_peered': '2024-09-15T21:25:54.456419+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:54.456419+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11153408, 'num_objects': 2723, 'num_object_clones': 0, 'num_object_copies': 5446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13294, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:57.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'15729", 'reported_seq': 16392, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:54.460744+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:54.460744+0000', 'last_peered': '2024-09-15T21:25:54.460744+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:54.460744+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10850304, 'num_objects': 2649, 'num_object_clones': 0, 'num_object_copies': 5298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13376, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -453}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:57.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:58.726 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:58.726 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23626/86248 objects degraded (27.393%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":571,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":43124,"data_bytes":176620180,"bytes_used":1648488448,"bytes_avail":288261804032,"bytes_total":289910292480,"degraded_objects":23626,"degraded_total":86248,"degraded_ratio":0.27393098970410906,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":499},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:25:58.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:58.770 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:25:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:25:59.779 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-15T21:25:58.508877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":176620180,"num_objects":43124,"num_object_clones":0,"num_object_copies":86248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224659,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16940,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609852,"kb_used_data":509688,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281505668,"statfs":{"total":289910292480,"available":288261804032,"internally_reserved":0,"allocated":521920512,"data_stored":520982109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,6,3,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19689472,"num_objects":-4807,"num_object_clones":0,"num_object_copies":-9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010533"},"pg_stats":[{"pgid":"2.f","version":"52'15805","reported_seq":16567,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.455786+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:54.455786+0000","last_peered":"2024-09-15T21:25:54.455786+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:54.455786+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087872,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15805,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'15936","reported_seq":16046,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.885493+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:54.885493+0000","last_peered":"2024-09-15T21:25:54.885493+0000","last_clean":"2024-09-15T21:25:54.885493+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:54.885493+0000","last_undegraded":"2024-09-15T21:25:54.885493+0000","last_fullsized":"2024-09-15T21:25:54.885493+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16009","reported_seq":22939,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.099926+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:25:55.099926+0000","last_peered":"2024-09-15T21:25:55.099926+0000","last_clean":"2024-09-15T21:25:55.099926+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:25:55.099926+0000","last_undegraded":"2024-09-15T21:25:55.099926+0000","last_fullsized":"2024-09-15T21:25:55.099926+0000","mapping_epoch":45,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11714584,"num_objects":2861,"num_object_clones":0,"num_object_copies":5722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14732,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'15934","reported_seq":16017,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.853711+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:54.853711+0000","last_peered":"2024-09-15T21:25:54.853711+0000","last_clean":"2024-09-15T21:25:54.853711+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:54.853711+0000","last_undegraded":"2024-09-15T21:25:54.853711+0000","last_fullsized":"2024-09-15T21:25:54.853711+0000","mapping_epoch":36,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'15991","reported_seq":16123,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.290371+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:54.290371+0000","last_peered":"2024-09-15T21:25:54.290371+0000","last_clean":"2024-09-15T21:25:54.290371+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:54.290371+0000","last_undegraded":"2024-09-15T21:25:54.290371+0000","last_fullsized":"2024-09-15T21:25:54.290371+0000","mapping_epoch":35,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11190272,"num_objects":2732,"num_object_clones":0,"num_object_copies":5464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15712","reported_seq":16539,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.417234+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:54.417234+0000","last_peered":"2024-09-15T21:25:54.417234+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:54.417234+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13137,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16191","reported_seq":16937,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.476152+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:54.476152+0000","last_peered":"2024-09-15T21:25:54.476152+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:54.476152+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15706","reported_seq":15839,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:54.398363+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:54.398363+0000","last_peered":"2024-09-15T21:25:54.398363+0000","last_clean":"2024-09-15T21:25:54.398363+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:54.398363+0000","last_undegraded":"2024-09-15T21:25:54.398363+0000","last_fullsized":"2024-09-15T21:25:54.398363+0000","mapping_epoch":35,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11051008,"num_objects":2698,"num_object_clones":0,"num_object_copies":5396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13491,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15684","reported_seq":22512,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.134391+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:25:55.134391+0000","last_peered":"2024-09-15T21:25:55.134391+0000","last_clean":"2024-09-15T21:25:55.134391+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:25:55.134391+0000","last_undegraded":"2024-09-15T21:25:55.134391+0000","last_fullsized":"2024-09-15T21:25:55.134391+0000","mapping_epoch":49,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727014+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727014+0000","last_peered":"2024-09-15T21:25:03.727014+0000","last_clean":"2024-09-15T21:25:03.727014+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727014+0000","last_undegraded":"2024-09-15T21:25:03.727014+0000","last_fullsized":"2024-09-15T21:25:03.727014+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15487","reported_seq":16276,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.450551+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:54.450551+0000","last_peered":"2024-09-15T21:25:54.450551+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:54.450551+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5400","ondisk_log_start":"44'5400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10432512,"num_objects":2547,"num_object_clones":0,"num_object_copies":5094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13013,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15584","reported_seq":18368,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.913303+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:54.913303+0000","last_peered":"2024-09-15T21:25:54.913303+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:54.913303+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5500","ondisk_log_start":"44'5500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063296,"num_objects":2701,"num_object_clones":0,"num_object_copies":5402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14135,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15821","reported_seq":16238,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.891327+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:54.891327+0000","last_peered":"2024-09-15T21:25:54.891327+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:54.891327+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11169792,"num_objects":2727,"num_object_clones":0,"num_object_copies":5454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14220,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-479}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'15987","reported_seq":16735,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.456419+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:54.456419+0000","last_peered":"2024-09-15T21:25:54.456419+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:54.456419+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11153408,"num_objects":2723,"num_object_clones":0,"num_object_copies":5446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15694","reported_seq":22612,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.115712+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:25:55.115712+0000","last_peered":"2024-09-15T21:25:55.115712+0000","last_clean":"2024-09-15T21:25:55.115712+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:25:55.115712+0000","last_undegraded":"2024-09-15T21:25:55.115712+0000","last_fullsized":"2024-09-15T21:25:55.115712+0000","mapping_epoch":51,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11132928,"num_objects":2718,"num_object_clones":0,"num_object_copies":5436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13279,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730284+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730284+0000","last_peered":"2024-09-15T21:25:03.730284+0000","last_clean":"2024-09-15T21:25:03.730284+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730284+0000","last_undegraded":"2024-09-15T21:25:03.730284+0000","last_fullsized":"2024-09-15T21:25:03.730284+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15729","reported_seq":16392,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:54.460744+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:54.460744+0000","last_peered":"2024-09-15T21:25:54.460744+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:54.460744+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10850304,"num_objects":2649,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'15973","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:55.062676+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:25:55.062676+0000","last_peered":"2024-09-15T21:25:55.062676+0000","last_clean":"2024-09-15T21:25:55.062676+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:25:55.062676+0000","last_undegraded":"2024-09-15T21:25:55.062676+0000","last_fullsized":"2024-09-15T21:25:55.062676+0000","mapping_epoch":47,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834537,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13633,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":176620161,"num_objects":43122,"num_object_clones":0,"num_object_copies":86244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23626,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":224657,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16939,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":518955008,"data_stored":518898158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953577,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484820,"kb_used_data":196288,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93887020,"statfs":{"total":96636764160,"available":96140308480,"internally_reserved":0,"allocated":200998912,"data_stored":200697510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149804,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575756,"kb_used_data":183152,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93796084,"statfs":{"total":96636764160,"available":96047190016,"internally_reserved":0,"allocated":187547648,"data_stored":187250318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549276,"kb_used_data":130248,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93822564,"statfs":{"total":96636764160,"available":96074305536,"internally_reserved":0,"allocated":133373952,"data_stored":133034281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,2,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":186580992,"data_stored":186577513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131973120,"data_stored":131927300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200400896,"data_stored":200393345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:02.781 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:03.240 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:03.241 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:03.296 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-15T21:26:02.509778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166797972,"num_objects":40726,"num_object_clones":0,"num_object_copies":81452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40726,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227057,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17063,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1590676,"kb_used_data":490532,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281524844,"statfs":{"total":289910292480,"available":288281440256,"internally_reserved":0,"allocated":502304768,"data_stored":501366365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,12,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19603456,"num_objects":-4786,"num_object_clones":0,"num_object_copies":-9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4786,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009704"},"pg_stats":[{"pgid":"2.f","version":"52'15934","reported_seq":16696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.467546+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:59.467546+0000","last_peered":"2024-09-15T21:25:59.467546+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:59.467546+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10559488,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16088","reported_seq":16198,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.812824+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:59.812824+0000","last_peered":"2024-09-15T21:25:59.812824+0000","last_clean":"2024-09-15T21:25:59.812824+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:59.812824+0000","last_undegraded":"2024-09-15T21:25:59.812824+0000","last_fullsized":"2024-09-15T21:25:59.812824+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14814,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16162","reported_seq":23092,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.139650+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:00.139650+0000","last_peered":"2024-09-15T21:26:00.139650+0000","last_clean":"2024-09-15T21:26:00.139650+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:00.139650+0000","last_undegraded":"2024-09-15T21:26:00.139650+0000","last_fullsized":"2024-09-15T21:26:00.139650+0000","mapping_epoch":45,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087896,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14885,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16084","reported_seq":16167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.943684+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:59.943684+0000","last_peered":"2024-09-15T21:25:59.943684+0000","last_clean":"2024-09-15T21:25:59.943684+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:59.943684+0000","last_undegraded":"2024-09-15T21:25:59.943684+0000","last_fullsized":"2024-09-15T21:25:59.943684+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16144","reported_seq":16276,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.457343+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:59.457343+0000","last_peered":"2024-09-15T21:25:59.457343+0000","last_clean":"2024-09-15T21:25:59.457343+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:59.457343+0000","last_undegraded":"2024-09-15T21:25:59.457343+0000","last_fullsized":"2024-09-15T21:25:59.457343+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10563584,"num_objects":2579,"num_object_clones":0,"num_object_copies":5158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15860","reported_seq":16687,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.438209+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:59.438209+0000","last_peered":"2024-09-15T21:25:59.438209+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:59.438209+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10326016,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13285,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-508}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16339","reported_seq":17090,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.453621+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:59.453621+0000","last_peered":"2024-09-15T21:25:59.453621+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:59.453621+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13975,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15864","reported_seq":15997,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.406076+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:59.406076+0000","last_peered":"2024-09-15T21:25:59.406076+0000","last_clean":"2024-09-15T21:25:59.406076+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:59.406076+0000","last_undegraded":"2024-09-15T21:25:59.406076+0000","last_fullsized":"2024-09-15T21:25:59.406076+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403840,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13649,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15824","reported_seq":22652,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.079796+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:00.079796+0000","last_peered":"2024-09-15T21:26:00.079796+0000","last_clean":"2024-09-15T21:26:00.079796+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:00.079796+0000","last_undegraded":"2024-09-15T21:26:00.079796+0000","last_fullsized":"2024-09-15T21:26:00.079796+0000","mapping_epoch":49,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10448896,"num_objects":2551,"num_object_clones":0,"num_object_copies":5102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14410,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15643","reported_seq":16432,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.465931+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:59.465931+0000","last_peered":"2024-09-15T21:25:59.465931+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:59.465931+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13169,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15737","reported_seq":18766,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.932936+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:59.932936+0000","last_peered":"2024-09-15T21:25:59.932936+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:59.932936+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14288,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15966","reported_seq":16383,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.896247+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:59.896247+0000","last_peered":"2024-09-15T21:25:59.896247+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:59.896247+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14365,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16141","reported_seq":16889,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.488632+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:59.488632+0000","last_peered":"2024-09-15T21:25:59.488632+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:59.488632+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522624,"num_objects":2569,"num_object_clones":0,"num_object_copies":5138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15840","reported_seq":22758,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.140370+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:00.140370+0000","last_peered":"2024-09-15T21:26:00.140370+0000","last_clean":"2024-09-15T21:26:00.140370+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:00.140370+0000","last_undegraded":"2024-09-15T21:26:00.140370+0000","last_fullsized":"2024-09-15T21:26:00.140370+0000","mapping_epoch":51,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15879","reported_seq":16543,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.437652+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:59.437652+0000","last_peered":"2024-09-15T21:25:59.437652+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:59.437652+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10235904,"num_objects":2499,"num_object_clones":0,"num_object_copies":4998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16136","reported_seq":23234,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.164045+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:00.164045+0000","last_peered":"2024-09-15T21:26:00.164045+0000","last_clean":"2024-09-15T21:26:00.164045+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:00.164045+0000","last_undegraded":"2024-09-15T21:26:00.164045+0000","last_fullsized":"2024-09-15T21:26:00.164045+0000","mapping_epoch":47,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166889,"num_objects":2483,"num_object_clones":0,"num_object_copies":4966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166797953,"num_objects":40724,"num_object_clones":0,"num_object_copies":81448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227055,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17062,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499339264,"data_stored":499282414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953578,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484132,"kb_used_data":191444,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93887708,"statfs":{"total":96636764160,"available":96141012992,"internally_reserved":0,"allocated":196038656,"data_stored":195737254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572540,"kb_used_data":175788,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93799300,"statfs":{"total":96636764160,"available":96050483200,"internally_reserved":0,"allocated":180006912,"data_stored":179709582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534004,"kb_used_data":123300,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93837836,"statfs":{"total":96636764160,"available":96089944064,"internally_reserved":0,"allocated":126259200,"data_stored":125919529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179179520,"data_stored":179176041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124739584,"data_stored":124693764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195420160,"data_stored":195412609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:03.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:04.110 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:04.111 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:04.164 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-15T21:26:02.509778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166797972,"num_objects":40726,"num_object_clones":0,"num_object_copies":81452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40726,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227057,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17063,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1590676,"kb_used_data":490532,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281524844,"statfs":{"total":289910292480,"available":288281440256,"internally_reserved":0,"allocated":502304768,"data_stored":501366365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,12,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19603456,"num_objects":-4786,"num_object_clones":0,"num_object_copies":-9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4786,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009704"},"pg_stats":[{"pgid":"2.f","version":"52'15934","reported_seq":16696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.467546+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:59.467546+0000","last_peered":"2024-09-15T21:25:59.467546+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:59.467546+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10559488,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16088","reported_seq":16198,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.812824+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:59.812824+0000","last_peered":"2024-09-15T21:25:59.812824+0000","last_clean":"2024-09-15T21:25:59.812824+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:59.812824+0000","last_undegraded":"2024-09-15T21:25:59.812824+0000","last_fullsized":"2024-09-15T21:25:59.812824+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14814,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16162","reported_seq":23092,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.139650+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:00.139650+0000","last_peered":"2024-09-15T21:26:00.139650+0000","last_clean":"2024-09-15T21:26:00.139650+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:00.139650+0000","last_undegraded":"2024-09-15T21:26:00.139650+0000","last_fullsized":"2024-09-15T21:26:00.139650+0000","mapping_epoch":45,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087896,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14885,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16084","reported_seq":16167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.943684+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:59.943684+0000","last_peered":"2024-09-15T21:25:59.943684+0000","last_clean":"2024-09-15T21:25:59.943684+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:59.943684+0000","last_undegraded":"2024-09-15T21:25:59.943684+0000","last_fullsized":"2024-09-15T21:25:59.943684+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16144","reported_seq":16276,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.457343+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:59.457343+0000","last_peered":"2024-09-15T21:25:59.457343+0000","last_clean":"2024-09-15T21:25:59.457343+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:59.457343+0000","last_undegraded":"2024-09-15T21:25:59.457343+0000","last_fullsized":"2024-09-15T21:25:59.457343+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10563584,"num_objects":2579,"num_object_clones":0,"num_object_copies":5158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15860","reported_seq":16687,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.438209+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:59.438209+0000","last_peered":"2024-09-15T21:25:59.438209+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:59.438209+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10326016,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13285,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-508}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16339","reported_seq":17090,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.453621+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:59.453621+0000","last_peered":"2024-09-15T21:25:59.453621+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:59.453621+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13975,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15864","reported_seq":15997,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.406076+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:59.406076+0000","last_peered":"2024-09-15T21:25:59.406076+0000","last_clean":"2024-09-15T21:25:59.406076+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:59.406076+0000","last_undegraded":"2024-09-15T21:25:59.406076+0000","last_fullsized":"2024-09-15T21:25:59.406076+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403840,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13649,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15824","reported_seq":22652,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.079796+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:00.079796+0000","last_peered":"2024-09-15T21:26:00.079796+0000","last_clean":"2024-09-15T21:26:00.079796+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:00.079796+0000","last_undegraded":"2024-09-15T21:26:00.079796+0000","last_fullsized":"2024-09-15T21:26:00.079796+0000","mapping_epoch":49,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10448896,"num_objects":2551,"num_object_clones":0,"num_object_copies":5102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14410,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15643","reported_seq":16432,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.465931+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:59.465931+0000","last_peered":"2024-09-15T21:25:59.465931+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:59.465931+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13169,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15737","reported_seq":18766,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.932936+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:59.932936+0000","last_peered":"2024-09-15T21:25:59.932936+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:59.932936+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14288,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15966","reported_seq":16383,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.896247+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:59.896247+0000","last_peered":"2024-09-15T21:25:59.896247+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:59.896247+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14365,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16141","reported_seq":16889,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.488632+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:59.488632+0000","last_peered":"2024-09-15T21:25:59.488632+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:59.488632+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522624,"num_objects":2569,"num_object_clones":0,"num_object_copies":5138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15840","reported_seq":22758,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.140370+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:00.140370+0000","last_peered":"2024-09-15T21:26:00.140370+0000","last_clean":"2024-09-15T21:26:00.140370+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:00.140370+0000","last_undegraded":"2024-09-15T21:26:00.140370+0000","last_fullsized":"2024-09-15T21:26:00.140370+0000","mapping_epoch":51,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15879","reported_seq":16543,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.437652+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:59.437652+0000","last_peered":"2024-09-15T21:25:59.437652+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:59.437652+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10235904,"num_objects":2499,"num_object_clones":0,"num_object_copies":4998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16136","reported_seq":23234,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.164045+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:00.164045+0000","last_peered":"2024-09-15T21:26:00.164045+0000","last_clean":"2024-09-15T21:26:00.164045+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:00.164045+0000","last_undegraded":"2024-09-15T21:26:00.164045+0000","last_fullsized":"2024-09-15T21:26:00.164045+0000","mapping_epoch":47,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166889,"num_objects":2483,"num_object_clones":0,"num_object_copies":4966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166797953,"num_objects":40724,"num_object_clones":0,"num_object_copies":81448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227055,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17062,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499339264,"data_stored":499282414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953578,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484132,"kb_used_data":191444,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93887708,"statfs":{"total":96636764160,"available":96141012992,"internally_reserved":0,"allocated":196038656,"data_stored":195737254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572540,"kb_used_data":175788,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93799300,"statfs":{"total":96636764160,"available":96050483200,"internally_reserved":0,"allocated":180006912,"data_stored":179709582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534004,"kb_used_data":123300,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93837836,"statfs":{"total":96636764160,"available":96089944064,"internally_reserved":0,"allocated":126259200,"data_stored":125919529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179179520,"data_stored":179176041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124739584,"data_stored":124693764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195420160,"data_stored":195412609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'15934", 'reported_seq': 16696, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.467546+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:25:59.467546+0000', 'last_peered': '2024-09-15T21:25:59.467546+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:25:59.467546+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10559488, 'num_objects': 2578, 'num_object_clones': 0, 'num_object_copies': 5156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15934, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -453}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'15860", 'reported_seq': 16687, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.438209+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:25:59.438209+0000', 'last_peered': '2024-09-15T21:25:59.438209+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:25:59.438209+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10326016, 'num_objects': 2521, 'num_object_clones': 0, 'num_object_copies': 5042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13285, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16339", 'reported_seq': 17090, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.453621+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:25:59.453621+0000', 'last_peered': '2024-09-15T21:25:59.453621+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:25:59.453621+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10506240, 'num_objects': 2565, 'num_object_clones': 0, 'num_object_copies': 5130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2565, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13975, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'15643", 'reported_seq': 16432, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.465931+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:25:59.465931+0000', 'last_peered': '2024-09-15T21:25:59.465931+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:25:59.465931+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'5600", 'ondisk_log_start': "44'5600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9793536, 'num_objects': 2391, 'num_object_clones': 0, 'num_object_copies': 4782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2391, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13169, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -624}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.166 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'15737", 'reported_seq': 18766, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.932936+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:25:59.932936+0000', 'last_peered': '2024-09-15T21:25:59.932936+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:25:59.932936+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'5700", 'ondisk_log_start': "44'5700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10436608, 'num_objects': 2548, 'num_object_clones': 0, 'num_object_copies': 5096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1815, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14288, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1453, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 733}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'15966", 'reported_seq': 16383, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.896247+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:25:59.896247+0000', 'last_peered': '2024-09-15T21:25:59.896247+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:25:59.896247+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10575872, 'num_objects': 2582, 'num_object_clones': 0, 'num_object_copies': 5164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14365, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -624}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'16141", 'reported_seq': 16889, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.488632+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:25:59.488632+0000', 'last_peered': '2024-09-15T21:25:59.488632+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:25:59.488632+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10522624, 'num_objects': 2569, 'num_object_clones': 0, 'num_object_copies': 5138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13448, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:04.167 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'15879", 'reported_seq': 16543, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:59.437652+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:25:59.437652+0000', 'last_peered': '2024-09-15T21:25:59.437652+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:25:59.437652+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'5800", 'ondisk_log_start': "44'5800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10235904, 'num_objects': 2499, 'num_object_clones': 0, 'num_object_copies': 4998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13526, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:04.167 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:05.184 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:05.185 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23503/81452 objects degraded (28.855%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":578,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40726,"data_bytes":166797972,"bytes_used":1628852224,"bytes_avail":288281440256,"bytes_total":289910292480,"degraded_objects":23503,"degraded_total":81452,"degraded_ratio":0.28855031184010216,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":398},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:26:05.255 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:05.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:05.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:26:05.645+0000 7f6caffff640 0 --2- 172.21.15.114:0/2772736242 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f6cb0073e50 0x7f6cb012d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:26:06.096 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:06.143 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-15T21:26:04.510331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166797972,"num_objects":40726,"num_object_clones":0,"num_object_copies":81452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40726,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227057,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17063,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1590676,"kb_used_data":490532,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281524844,"statfs":{"total":289910292480,"available":288281440256,"internally_reserved":0,"allocated":502304768,"data_stored":501366365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,12,6,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19603456,"num_objects":-4786,"num_object_clones":0,"num_object_copies":-9572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4786,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005887"},"pg_stats":[{"pgid":"2.f","version":"52'15934","reported_seq":16696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.467546+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:25:59.467546+0000","last_peered":"2024-09-15T21:25:59.467546+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:25:59.467546+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10559488,"num_objects":2578,"num_object_clones":0,"num_object_copies":5156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15934,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16088","reported_seq":16198,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.812824+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:25:59.812824+0000","last_peered":"2024-09-15T21:25:59.812824+0000","last_clean":"2024-09-15T21:25:59.812824+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:25:59.812824+0000","last_undegraded":"2024-09-15T21:25:59.812824+0000","last_fullsized":"2024-09-15T21:25:59.812824+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10452992,"num_objects":2552,"num_object_clones":0,"num_object_copies":5104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14814,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16162","reported_seq":23092,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.139650+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:00.139650+0000","last_peered":"2024-09-15T21:26:00.139650+0000","last_clean":"2024-09-15T21:26:00.139650+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:00.139650+0000","last_undegraded":"2024-09-15T21:26:00.139650+0000","last_fullsized":"2024-09-15T21:26:00.139650+0000","mapping_epoch":45,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11087896,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14885,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16084","reported_seq":16167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.943684+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:25:59.943684+0000","last_peered":"2024-09-15T21:25:59.943684+0000","last_clean":"2024-09-15T21:25:59.943684+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:25:59.943684+0000","last_undegraded":"2024-09-15T21:25:59.943684+0000","last_fullsized":"2024-09-15T21:25:59.943684+0000","mapping_epoch":36,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10182656,"num_objects":2486,"num_object_clones":0,"num_object_copies":4972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16144","reported_seq":16276,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.457343+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:25:59.457343+0000","last_peered":"2024-09-15T21:25:59.457343+0000","last_clean":"2024-09-15T21:25:59.457343+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:25:59.457343+0000","last_undegraded":"2024-09-15T21:25:59.457343+0000","last_fullsized":"2024-09-15T21:25:59.457343+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10563584,"num_objects":2579,"num_object_clones":0,"num_object_copies":5158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14002,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'15860","reported_seq":16687,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.438209+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:25:59.438209+0000","last_peered":"2024-09-15T21:25:59.438209+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:25:59.438209+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10326016,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13285,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-508}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16339","reported_seq":17090,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.453621+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:25:59.453621+0000","last_peered":"2024-09-15T21:25:59.453621+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:25:59.453621+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506240,"num_objects":2565,"num_object_clones":0,"num_object_copies":5130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13975,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'15864","reported_seq":15997,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:59.406076+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:25:59.406076+0000","last_peered":"2024-09-15T21:25:59.406076+0000","last_clean":"2024-09-15T21:25:59.406076+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:25:59.406076+0000","last_undegraded":"2024-09-15T21:25:59.406076+0000","last_fullsized":"2024-09-15T21:25:59.406076+0000","mapping_epoch":35,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10403840,"num_objects":2540,"num_object_clones":0,"num_object_copies":5080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13649,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15824","reported_seq":22652,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.079796+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:00.079796+0000","last_peered":"2024-09-15T21:26:00.079796+0000","last_clean":"2024-09-15T21:26:00.079796+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:00.079796+0000","last_undegraded":"2024-09-15T21:26:00.079796+0000","last_fullsized":"2024-09-15T21:26:00.079796+0000","mapping_epoch":49,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10448896,"num_objects":2551,"num_object_clones":0,"num_object_copies":5102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14410,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15643","reported_seq":16432,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.465931+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:25:59.465931+0000","last_peered":"2024-09-15T21:25:59.465931+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:25:59.465931+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5600","ondisk_log_start":"44'5600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9793536,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13169,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15737","reported_seq":18766,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.932936+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:25:59.932936+0000","last_peered":"2024-09-15T21:25:59.932936+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:25:59.932936+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14288,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":733}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'15966","reported_seq":16383,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.896247+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:25:59.896247+0000","last_peered":"2024-09-15T21:25:59.896247+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:25:59.896247+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10575872,"num_objects":2582,"num_object_clones":0,"num_object_copies":5164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14365,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16141","reported_seq":16889,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.488632+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:25:59.488632+0000","last_peered":"2024-09-15T21:25:59.488632+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:25:59.488632+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522624,"num_objects":2569,"num_object_clones":0,"num_object_copies":5138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-561}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15840","reported_seq":22758,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.140370+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:00.140370+0000","last_peered":"2024-09-15T21:26:00.140370+0000","last_clean":"2024-09-15T21:26:00.140370+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:00.140370+0000","last_undegraded":"2024-09-15T21:26:00.140370+0000","last_fullsized":"2024-09-15T21:26:00.140370+0000","mapping_epoch":51,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'15879","reported_seq":16543,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:59.437652+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:25:59.437652+0000","last_peered":"2024-09-15T21:25:59.437652+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:25:59.437652+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10235904,"num_objects":2499,"num_object_clones":0,"num_object_copies":4998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16136","reported_seq":23234,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:00.164045+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:00.164045+0000","last_peered":"2024-09-15T21:26:00.164045+0000","last_clean":"2024-09-15T21:26:00.164045+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:00.164045+0000","last_undegraded":"2024-09-15T21:26:00.164045+0000","last_fullsized":"2024-09-15T21:26:00.164045+0000","mapping_epoch":47,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166889,"num_objects":2483,"num_object_clones":0,"num_object_copies":4966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166797953,"num_objects":40724,"num_object_clones":0,"num_object_copies":81448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227055,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17062,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499339264,"data_stored":499282414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953578,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484132,"kb_used_data":191444,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93887708,"statfs":{"total":96636764160,"available":96141012992,"internally_reserved":0,"allocated":196038656,"data_stored":195737254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149805,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572540,"kb_used_data":175788,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93799300,"statfs":{"total":96636764160,"available":96050483200,"internally_reserved":0,"allocated":180006912,"data_stored":179709582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626524,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534004,"kb_used_data":123300,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93837836,"statfs":{"total":96636764160,"available":96089944064,"internally_reserved":0,"allocated":126259200,"data_stored":125919529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179179520,"data_stored":179176041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124739584,"data_stored":124693764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195420160,"data_stored":195412609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:09.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:09.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:10.000 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:10.079 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-15T21:26:08.511235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":157172372,"num_objects":38376,"num_object_clones":0,"num_object_copies":76752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229407,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17203,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572020,"kb_used_data":471868,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281543500,"statfs":{"total":289910292480,"available":288300544000,"internally_reserved":0,"allocated":483192832,"data_stored":482254429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,4,9,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19447808,"num_objects":-4748,"num_object_clones":0,"num_object_copies":-9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002819"},"pg_stats":[{"pgid":"2.f","version":"52'16071","reported_seq":16833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.465742+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:04.465742+0000","last_peered":"2024-09-15T21:26:04.465742+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:04.465742+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9998336,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16071,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16234","reported_seq":16344,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:04.836812+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:04.836812+0000","last_peered":"2024-09-15T21:26:04.836812+0000","last_clean":"2024-09-15T21:26:04.836812+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:04.836812+0000","last_undegraded":"2024-09-15T21:26:04.836812+0000","last_fullsized":"2024-09-15T21:26:04.836812+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9854976,"num_objects":2406,"num_object_clones":0,"num_object_copies":4812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14960,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16315","reported_seq":23245,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:05.244656+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:05.244656+0000","last_peered":"2024-09-15T21:26:05.244656+0000","last_clean":"2024-09-15T21:26:05.244656+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:05.244656+0000","last_undegraded":"2024-09-15T21:26:05.244656+0000","last_fullsized":"2024-09-15T21:26:05.244656+0000","mapping_epoch":45,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10461208,"num_objects":2555,"num_object_clones":0,"num_object_copies":5110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15038,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16223","reported_seq":16306,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:04.939134+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:04.939134+0000","last_peered":"2024-09-15T21:26:04.939134+0000","last_clean":"2024-09-15T21:26:04.939134+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:04.939134+0000","last_undegraded":"2024-09-15T21:26:04.939134+0000","last_fullsized":"2024-09-15T21:26:04.939134+0000","mapping_epoch":36,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9613312,"num_objects":2347,"num_object_clones":0,"num_object_copies":4694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16304","reported_seq":16436,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:04.454117+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:04.454117+0000","last_peered":"2024-09-15T21:26:04.454117+0000","last_clean":"2024-09-15T21:26:04.454117+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:04.454117+0000","last_undegraded":"2024-09-15T21:26:04.454117+0000","last_fullsized":"2024-09-15T21:26:04.454117+0000","mapping_epoch":35,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9908224,"num_objects":2419,"num_object_clones":0,"num_object_copies":4838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14162,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16020","reported_seq":16847,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.411098+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:04.411098+0000","last_peered":"2024-09-15T21:26:04.411098+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:04.411098+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9670656,"num_objects":2361,"num_object_clones":0,"num_object_copies":4722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13445,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16475","reported_seq":17226,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.428548+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:04.428548+0000","last_peered":"2024-09-15T21:26:04.428548+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:04.428548+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9949184,"num_objects":2429,"num_object_clones":0,"num_object_copies":4858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14111,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16006","reported_seq":16139,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:04.444172+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:04.444172+0000","last_peered":"2024-09-15T21:26:04.444172+0000","last_clean":"2024-09-15T21:26:04.444172+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:04.444172+0000","last_undegraded":"2024-09-15T21:26:04.444172+0000","last_fullsized":"2024-09-15T21:26:04.444172+0000","mapping_epoch":35,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9822208,"num_objects":2398,"num_object_clones":0,"num_object_copies":4796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13791,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'15965","reported_seq":22791,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:05.226422+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:05.226422+0000","last_peered":"2024-09-15T21:26:05.226422+0000","last_clean":"2024-09-15T21:26:05.226422+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:05.226422+0000","last_undegraded":"2024-09-15T21:26:05.226422+0000","last_fullsized":"2024-09-15T21:26:05.226422+0000","mapping_epoch":49,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9871360,"num_objects":2410,"num_object_clones":0,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14551,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15774","reported_seq":16563,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.394996+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:04.394996+0000","last_peered":"2024-09-15T21:26:04.394996+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:04.394996+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5700","ondisk_log_start":"44'5700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9256960,"num_objects":2260,"num_object_clones":0,"num_object_copies":4520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13300,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'15870","reported_seq":19180,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.924694+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:04.924694+0000","last_peered":"2024-09-15T21:26:04.924694+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:04.924694+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'5800","ondisk_log_start":"44'5800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9891840,"num_objects":2415,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14421,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1593,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":740}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16124","reported_seq":16540,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.945951+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:04.945951+0000","last_peered":"2024-09-15T21:26:04.945951+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:04.945951+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9928704,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14523,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16289","reported_seq":17035,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.510773+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:04.510773+0000","last_peered":"2024-09-15T21:26:04.510773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:04.510773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9916416,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13596,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'15999","reported_seq":22917,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:05.242685+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:05.242685+0000","last_peered":"2024-09-15T21:26:05.242685+0000","last_clean":"2024-09-15T21:26:05.242685+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:05.242685+0000","last_undegraded":"2024-09-15T21:26:05.242685+0000","last_fullsized":"2024-09-15T21:26:05.242685+0000","mapping_epoch":51,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9883648,"num_objects":2413,"num_object_clones":0,"num_object_copies":4826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13584,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16016","reported_seq":16680,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:04.476691+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:04.476691+0000","last_peered":"2024-09-15T21:26:04.476691+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:04.476691+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9674752,"num_objects":2362,"num_object_clones":0,"num_object_copies":4724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13663,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16306","reported_seq":23403,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:05.232583+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:05.232583+0000","last_peered":"2024-09-15T21:26:05.232583+0000","last_clean":"2024-09-15T21:26:05.232583+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:05.232583+0000","last_undegraded":"2024-09-15T21:26:05.232583+0000","last_fullsized":"2024-09-15T21:26:05.232583+0000","mapping_epoch":47,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9470569,"num_objects":2313,"num_object_clones":0,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13966,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":157172353,"num_objects":38374,"num_object_clones":0,"num_object_copies":76748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229405,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17202,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":480096256,"data_stored":480039406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953579,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467000,"kb_used_data":186788,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93904840,"statfs":{"total":96636764160,"available":96158556160,"internally_reserved":0,"allocated":191270912,"data_stored":190969510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573952,"kb_used_data":168880,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93797888,"statfs":{"total":96636764160,"available":96049037312,"internally_reserved":0,"allocated":172933120,"data_stored":172635790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":414772913},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531068,"kb_used_data":116200,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93840772,"statfs":{"total":96636764160,"available":96092950528,"internally_reserved":0,"allocated":118988800,"data_stored":118649129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":424799916},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,2,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171995136,"data_stored":171991657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117366784,"data_stored":117320964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":190734336,"data_stored":190726785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:10.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:10.635 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:10.635 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:10.663 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T21:26:10.511721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147317396,"num_objects":35970,"num_object_clones":0,"num_object_copies":71940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231813,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17312,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536220,"kb_used_data":452716,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281579300,"statfs":{"total":289910292480,"available":288337203200,"internally_reserved":0,"allocated":463581184,"data_stored":462642781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1109445642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,1,6,7,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29302784,"num_objects":-7154,"num_object_clones":0,"num_object_copies":-14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"52'16194","reported_seq":16955,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.562423+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:09.562423+0000","last_peered":"2024-09-15T21:26:09.562423+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:09.562423+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9494528,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-713}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16394","reported_seq":16504,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.973086+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:09.973086+0000","last_peered":"2024-09-15T21:26:09.973086+0000","last_clean":"2024-09-15T21:26:09.973086+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:09.973086+0000","last_undegraded":"2024-09-15T21:26:09.973086+0000","last_fullsized":"2024-09-15T21:26:09.973086+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15120,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16469","reported_seq":23396,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.246913+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:10.246913+0000","last_peered":"2024-09-15T21:26:10.246913+0000","last_clean":"2024-09-15T21:26:10.246913+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:10.246913+0000","last_undegraded":"2024-09-15T21:26:10.246913+0000","last_fullsized":"2024-09-15T21:26:10.246913+0000","mapping_epoch":45,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9830424,"num_objects":2401,"num_object_clones":0,"num_object_copies":4802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15192,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16378","reported_seq":16461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.843961+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:09.843961+0000","last_peered":"2024-09-15T21:26:09.843961+0000","last_clean":"2024-09-15T21:26:09.843961+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:09.843961+0000","last_undegraded":"2024-09-15T21:26:09.843961+0000","last_fullsized":"2024-09-15T21:26:09.843961+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8978432,"num_objects":2192,"num_object_clones":0,"num_object_copies":4384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16378,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16451","reported_seq":16583,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.543611+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:09.543611+0000","last_peered":"2024-09-15T21:26:09.543611+0000","last_clean":"2024-09-15T21:26:09.543611+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:09.543611+0000","last_undegraded":"2024-09-15T21:26:09.543611+0000","last_fullsized":"2024-09-15T21:26:09.543611+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16187","reported_seq":17014,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.495119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:09.495119+0000","last_peered":"2024-09-15T21:26:09.495119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:09.495119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8986624,"num_objects":2194,"num_object_clones":0,"num_object_copies":4388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16632","reported_seq":17380,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.531281+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:09.531281+0000","last_peered":"2024-09-15T21:26:09.531281+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:09.531281+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16150","reported_seq":16284,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.454017+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:09.454017+0000","last_peered":"2024-09-15T21:26:09.454017+0000","last_clean":"2024-09-15T21:26:09.454017+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:09.454017+0000","last_undegraded":"2024-09-15T21:26:09.454017+0000","last_fullsized":"2024-09-15T21:26:09.454017+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13935,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16130","reported_seq":22958,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.233878+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:10.233878+0000","last_peered":"2024-09-15T21:26:10.233878+0000","last_clean":"2024-09-15T21:26:10.233878+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:10.233878+0000","last_undegraded":"2024-09-15T21:26:10.233878+0000","last_fullsized":"2024-09-15T21:26:10.233878+0000","mapping_epoch":49,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9195520,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14716,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15916","reported_seq":16704,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.564612+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:09.564612+0000","last_peered":"2024-09-15T21:26:09.564612+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:09.564612+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8675328,"num_objects":2118,"num_object_clones":0,"num_object_copies":4236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13442,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16029","reported_seq":19556,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:10.000228+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:10.000228+0000","last_peered":"2024-09-15T21:26:10.000228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:10.000228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9240576,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":690}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16281","reported_seq":16698,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.953684+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:09.953684+0000","last_peered":"2024-09-15T21:26:09.953684+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:09.953684+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285632,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16428","reported_seq":17176,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.521530+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:09.521530+0000","last_peered":"2024-09-15T21:26:09.521530+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:09.521530+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16153","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.200163+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:10.200163+0000","last_peered":"2024-09-15T21:26:10.200163+0000","last_clean":"2024-09-15T21:26:10.200163+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:10.200163+0000","last_undegraded":"2024-09-15T21:26:10.200163+0000","last_fullsized":"2024-09-15T21:26:10.200163+0000","mapping_epoch":51,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9252864,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13738,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16168","reported_seq":16832,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.555801+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:09.555801+0000","last_peered":"2024-09-15T21:26:09.555801+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:09.555801+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9052160,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16437","reported_seq":23535,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.251504+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:10.251504+0000","last_peered":"2024-09-15T21:26:10.251504+0000","last_clean":"2024-09-15T21:26:10.251504+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:10.251504+0000","last_undegraded":"2024-09-15T21:26:10.251504+0000","last_fullsized":"2024-09-15T21:26:10.251504+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8933993,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14097,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":147317377,"num_objects":35968,"num_object_clones":0,"num_object_copies":71936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231811,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17311,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":460255232,"data_stored":460198382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466228,"kb_used_data":181860,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93905612,"statfs":{"total":96636764160,"available":96159346688,"internally_reserved":0,"allocated":186224640,"data_stored":185923238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149807,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554404,"kb_used_data":161812,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93817436,"statfs":{"total":96636764160,"available":96069054464,"internally_reserved":0,"allocated":165695488,"data_stored":165398158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515588,"kb_used_data":109044,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93856252,"statfs":{"total":96636764160,"available":96108802048,"internally_reserved":0,"allocated":111661056,"data_stored":111321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":416280236},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164646912,"data_stored":164643433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109993984,"data_stored":109948164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185614336,"data_stored":185606785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:10.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:10.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'16194", 'reported_seq': 16955, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.562423+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:09.562423+0000', 'last_peered': '2024-09-15T21:26:09.562423+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:09.562423+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9494528, 'num_objects': 2318, 'num_object_clones': 0, 'num_object_copies': 4636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16194, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -713}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'16187", 'reported_seq': 17014, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.495119+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:09.495119+0000', 'last_peered': '2024-09-15T21:26:09.495119+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:09.495119+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8986624, 'num_objects': 2194, 'num_object_clones': 0, 'num_object_copies': 4388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13612, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16632", 'reported_seq': 17380, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.531281+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:09.531281+0000', 'last_peered': '2024-09-15T21:26:09.531281+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:09.531281+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'6600", 'ondisk_log_start': "46'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9306112, 'num_objects': 2272, 'num_object_clones': 0, 'num_object_copies': 4544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14268, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'15916", 'reported_seq': 16704, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.564612+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:09.564612+0000', 'last_peered': '2024-09-15T21:26:09.564612+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:09.564612+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'5900", 'ondisk_log_start': "44'5900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8675328, 'num_objects': 2118, 'num_object_clones': 0, 'num_object_copies': 4236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13442, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -897}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16029", 'reported_seq': 19556, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:10.000228+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:10.000228+0000', 'last_peered': '2024-09-15T21:26:10.000228+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:10.000228+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9240576, 'num_objects': 2256, 'num_object_clones': 0, 'num_object_copies': 4512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1566, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14580, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1702, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 690}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:10.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'16281", 'reported_seq': 16698, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.953684+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:09.953684+0000', 'last_peered': '2024-09-15T21:26:09.953684+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:09.953684+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "44'6200", 'ondisk_log_start': "44'6200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9285632, 'num_objects': 2267, 'num_object_clones': 0, 'num_object_copies': 4534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14680, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:10.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:10.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'16428", 'reported_seq': 17176, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.521530+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:09.521530+0000', 'last_peered': '2024-09-15T21:26:09.521530+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:09.521530+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "44'6400", 'ondisk_log_start': "44'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9347072, 'num_objects': 2282, 'num_object_clones': 0, 'num_object_copies': 4564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13735, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -848}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:10.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'16168", 'reported_seq': 16832, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:09.555801+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:09.555801+0000', 'last_peered': '2024-09-15T21:26:09.555801+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:09.555801+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9052160, 'num_objects': 2210, 'num_object_clones': 0, 'num_object_copies': 4420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13815, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:10.666 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:11.497 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:11.497 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23254/71940 objects degraded (32.324%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":584,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":35970,"data_bytes":147317396,"bytes_used":1573089280,"bytes_avail":288337203200,"bytes_total":289910292480,"degraded_objects":23254,"degraded_total":71940,"degraded_ratio":0.32324159021406729,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":372,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":596},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:26:11.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:11.515 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:11.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:26:11.857+0000 7f4d85c31640 0 --2- 172.21.15.114:0/1956495822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4d8012b260 0x7f4d80091b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:26:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:12.350 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:12.400 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T21:26:10.511721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147317396,"num_objects":35970,"num_object_clones":0,"num_object_copies":71940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231813,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17312,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536220,"kb_used_data":452716,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281579300,"statfs":{"total":289910292480,"available":288337203200,"internally_reserved":0,"allocated":463581184,"data_stored":462642781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1109445642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,1,6,7,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29302784,"num_objects":-7154,"num_object_clones":0,"num_object_copies":-14308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7154,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"52'16194","reported_seq":16955,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.562423+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:09.562423+0000","last_peered":"2024-09-15T21:26:09.562423+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:09.562423+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9494528,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-713}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16394","reported_seq":16504,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.973086+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:09.973086+0000","last_peered":"2024-09-15T21:26:09.973086+0000","last_clean":"2024-09-15T21:26:09.973086+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:09.973086+0000","last_undegraded":"2024-09-15T21:26:09.973086+0000","last_fullsized":"2024-09-15T21:26:09.973086+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15120,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16469","reported_seq":23396,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.246913+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:10.246913+0000","last_peered":"2024-09-15T21:26:10.246913+0000","last_clean":"2024-09-15T21:26:10.246913+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:10.246913+0000","last_undegraded":"2024-09-15T21:26:10.246913+0000","last_fullsized":"2024-09-15T21:26:10.246913+0000","mapping_epoch":45,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9830424,"num_objects":2401,"num_object_clones":0,"num_object_copies":4802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15192,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16378","reported_seq":16461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.843961+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:09.843961+0000","last_peered":"2024-09-15T21:26:09.843961+0000","last_clean":"2024-09-15T21:26:09.843961+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:09.843961+0000","last_undegraded":"2024-09-15T21:26:09.843961+0000","last_fullsized":"2024-09-15T21:26:09.843961+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8978432,"num_objects":2192,"num_object_clones":0,"num_object_copies":4384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16378,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16451","reported_seq":16583,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.543611+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:09.543611+0000","last_peered":"2024-09-15T21:26:09.543611+0000","last_clean":"2024-09-15T21:26:09.543611+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:09.543611+0000","last_undegraded":"2024-09-15T21:26:09.543611+0000","last_fullsized":"2024-09-15T21:26:09.543611+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16187","reported_seq":17014,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.495119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:09.495119+0000","last_peered":"2024-09-15T21:26:09.495119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:09.495119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8986624,"num_objects":2194,"num_object_clones":0,"num_object_copies":4388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16632","reported_seq":17380,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.531281+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:09.531281+0000","last_peered":"2024-09-15T21:26:09.531281+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:09.531281+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16150","reported_seq":16284,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.454017+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:09.454017+0000","last_peered":"2024-09-15T21:26:09.454017+0000","last_clean":"2024-09-15T21:26:09.454017+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:09.454017+0000","last_undegraded":"2024-09-15T21:26:09.454017+0000","last_fullsized":"2024-09-15T21:26:09.454017+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13935,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16130","reported_seq":22958,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.233878+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:10.233878+0000","last_peered":"2024-09-15T21:26:10.233878+0000","last_clean":"2024-09-15T21:26:10.233878+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:10.233878+0000","last_undegraded":"2024-09-15T21:26:10.233878+0000","last_fullsized":"2024-09-15T21:26:10.233878+0000","mapping_epoch":49,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9195520,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14716,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15916","reported_seq":16704,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.564612+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:09.564612+0000","last_peered":"2024-09-15T21:26:09.564612+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:09.564612+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8675328,"num_objects":2118,"num_object_clones":0,"num_object_copies":4236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13442,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16029","reported_seq":19556,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:10.000228+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:10.000228+0000","last_peered":"2024-09-15T21:26:10.000228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:10.000228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9240576,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":690}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16281","reported_seq":16698,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.953684+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:09.953684+0000","last_peered":"2024-09-15T21:26:09.953684+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:09.953684+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285632,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16428","reported_seq":17176,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.521530+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:09.521530+0000","last_peered":"2024-09-15T21:26:09.521530+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:09.521530+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16153","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.200163+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:10.200163+0000","last_peered":"2024-09-15T21:26:10.200163+0000","last_clean":"2024-09-15T21:26:10.200163+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:10.200163+0000","last_undegraded":"2024-09-15T21:26:10.200163+0000","last_fullsized":"2024-09-15T21:26:10.200163+0000","mapping_epoch":51,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9252864,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13738,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16168","reported_seq":16832,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.555801+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:09.555801+0000","last_peered":"2024-09-15T21:26:09.555801+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:09.555801+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9052160,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16437","reported_seq":23535,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.251504+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:10.251504+0000","last_peered":"2024-09-15T21:26:10.251504+0000","last_clean":"2024-09-15T21:26:10.251504+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:10.251504+0000","last_undegraded":"2024-09-15T21:26:10.251504+0000","last_fullsized":"2024-09-15T21:26:10.251504+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8933993,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14097,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":147317377,"num_objects":35968,"num_object_clones":0,"num_object_copies":71936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231811,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17311,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":460255232,"data_stored":460198382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466228,"kb_used_data":181860,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93905612,"statfs":{"total":96636764160,"available":96159346688,"internally_reserved":0,"allocated":186224640,"data_stored":185923238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149807,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554404,"kb_used_data":161812,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93817436,"statfs":{"total":96636764160,"available":96069054464,"internally_reserved":0,"allocated":165695488,"data_stored":165398158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515588,"kb_used_data":109044,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93856252,"statfs":{"total":96636764160,"available":96108802048,"internally_reserved":0,"allocated":111661056,"data_stored":111321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":416280236},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164646912,"data_stored":164643433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109993984,"data_stored":109948164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185614336,"data_stored":185606785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:15.404 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:16.263 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:16.268 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:16.338 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-15T21:26:14.512587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147317396,"num_objects":35970,"num_object_clones":0,"num_object_copies":71940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231813,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17312,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536220,"kb_used_data":452716,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281579300,"statfs":{"total":289910292480,"available":288337203200,"internally_reserved":0,"allocated":463581184,"data_stored":462642781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1109445642},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,1,6,7,12,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19480576,"num_objects":-4756,"num_object_clones":0,"num_object_copies":-9512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4756,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002808"},"pg_stats":[{"pgid":"2.f","version":"52'16194","reported_seq":16955,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.562423+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:09.562423+0000","last_peered":"2024-09-15T21:26:09.562423+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:09.562423+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9494528,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-713}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16394","reported_seq":16504,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.973086+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:09.973086+0000","last_peered":"2024-09-15T21:26:09.973086+0000","last_clean":"2024-09-15T21:26:09.973086+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:09.973086+0000","last_undegraded":"2024-09-15T21:26:09.973086+0000","last_fullsized":"2024-09-15T21:26:09.973086+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15120,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16469","reported_seq":23396,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.246913+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:10.246913+0000","last_peered":"2024-09-15T21:26:10.246913+0000","last_clean":"2024-09-15T21:26:10.246913+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:10.246913+0000","last_undegraded":"2024-09-15T21:26:10.246913+0000","last_fullsized":"2024-09-15T21:26:10.246913+0000","mapping_epoch":45,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9830424,"num_objects":2401,"num_object_clones":0,"num_object_copies":4802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15192,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16378","reported_seq":16461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.843961+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:09.843961+0000","last_peered":"2024-09-15T21:26:09.843961+0000","last_clean":"2024-09-15T21:26:09.843961+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:09.843961+0000","last_undegraded":"2024-09-15T21:26:09.843961+0000","last_fullsized":"2024-09-15T21:26:09.843961+0000","mapping_epoch":36,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8978432,"num_objects":2192,"num_object_clones":0,"num_object_copies":4384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16378,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16451","reported_seq":16583,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.543611+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:09.543611+0000","last_peered":"2024-09-15T21:26:09.543611+0000","last_clean":"2024-09-15T21:26:09.543611+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:09.543611+0000","last_undegraded":"2024-09-15T21:26:09.543611+0000","last_fullsized":"2024-09-15T21:26:09.543611+0000","mapping_epoch":35,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16187","reported_seq":17014,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.495119+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:09.495119+0000","last_peered":"2024-09-15T21:26:09.495119+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:09.495119+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8986624,"num_objects":2194,"num_object_clones":0,"num_object_copies":4388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16632","reported_seq":17380,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.531281+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:09.531281+0000","last_peered":"2024-09-15T21:26:09.531281+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:09.531281+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16150","reported_seq":16284,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:09.454017+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:09.454017+0000","last_peered":"2024-09-15T21:26:09.454017+0000","last_clean":"2024-09-15T21:26:09.454017+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:09.454017+0000","last_undegraded":"2024-09-15T21:26:09.454017+0000","last_fullsized":"2024-09-15T21:26:09.454017+0000","mapping_epoch":35,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13935,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16130","reported_seq":22958,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.233878+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:10.233878+0000","last_peered":"2024-09-15T21:26:10.233878+0000","last_clean":"2024-09-15T21:26:10.233878+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:10.233878+0000","last_undegraded":"2024-09-15T21:26:10.233878+0000","last_fullsized":"2024-09-15T21:26:10.233878+0000","mapping_epoch":49,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9195520,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14716,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'15916","reported_seq":16704,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.564612+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:09.564612+0000","last_peered":"2024-09-15T21:26:09.564612+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:09.564612+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'5900","ondisk_log_start":"44'5900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8675328,"num_objects":2118,"num_object_clones":0,"num_object_copies":4236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13442,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16029","reported_seq":19556,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:10.000228+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:10.000228+0000","last_peered":"2024-09-15T21:26:10.000228+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:10.000228+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9240576,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":690}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16281","reported_seq":16698,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.953684+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:09.953684+0000","last_peered":"2024-09-15T21:26:09.953684+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:09.953684+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285632,"num_objects":2267,"num_object_clones":0,"num_object_copies":4534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16428","reported_seq":17176,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.521530+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:09.521530+0000","last_peered":"2024-09-15T21:26:09.521530+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:09.521530+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9347072,"num_objects":2282,"num_object_clones":0,"num_object_copies":4564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13735,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16153","reported_seq":23071,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.200163+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:10.200163+0000","last_peered":"2024-09-15T21:26:10.200163+0000","last_clean":"2024-09-15T21:26:10.200163+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:10.200163+0000","last_undegraded":"2024-09-15T21:26:10.200163+0000","last_fullsized":"2024-09-15T21:26:10.200163+0000","mapping_epoch":51,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9252864,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13738,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16168","reported_seq":16832,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:09.555801+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:09.555801+0000","last_peered":"2024-09-15T21:26:09.555801+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:09.555801+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9052160,"num_objects":2210,"num_object_clones":0,"num_object_copies":4420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13815,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16437","reported_seq":23535,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:10.251504+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:10.251504+0000","last_peered":"2024-09-15T21:26:10.251504+0000","last_clean":"2024-09-15T21:26:10.251504+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:10.251504+0000","last_undegraded":"2024-09-15T21:26:10.251504+0000","last_fullsized":"2024-09-15T21:26:10.251504+0000","mapping_epoch":47,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8933993,"num_objects":2182,"num_object_clones":0,"num_object_copies":4364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14097,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":147317377,"num_objects":35968,"num_object_clones":0,"num_object_copies":71936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231811,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17311,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":460255232,"data_stored":460198382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466228,"kb_used_data":181860,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93905612,"statfs":{"total":96636764160,"available":96159346688,"internally_reserved":0,"allocated":186224640,"data_stored":185923238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149807,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554404,"kb_used_data":161812,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93817436,"statfs":{"total":96636764160,"available":96069054464,"internally_reserved":0,"allocated":165695488,"data_stored":165398158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":401993393},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515588,"kb_used_data":109044,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93856252,"statfs":{"total":96636764160,"available":96108802048,"internally_reserved":0,"allocated":111661056,"data_stored":111321385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":416280236},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164646912,"data_stored":164643433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109993984,"data_stored":109948164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185614336,"data_stored":185606785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:16.339 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:17.148 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:17.150 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:17.185 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-15T21:26:16.512927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137978516,"num_objects":33690,"num_object_clones":0,"num_object_copies":67380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234093,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17467,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160880,"ondisk_log_size":160880,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534776,"kb_used_data":434620,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281580744,"statfs":{"total":289910292480,"available":288338681856,"internally_reserved":0,"allocated":445050880,"data_stored":444112477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,4,6,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28819456,"num_objects":-7036,"num_object_clones":0,"num_object_copies":-14072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7036,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002596"},"pg_stats":[{"pgid":"2.f","version":"52'16334","reported_seq":17095,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.531847+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:14.531847+0000","last_peered":"2024-09-15T21:26:14.531847+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:14.531847+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8921088,"num_objects":2178,"num_object_clones":0,"num_object_copies":4356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16520","reported_seq":16630,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.953699+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:14.953699+0000","last_peered":"2024-09-15T21:26:14.953699+0000","last_clean":"2024-09-15T21:26:14.953699+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:14.953699+0000","last_undegraded":"2024-09-15T21:26:14.953699+0000","last_fullsized":"2024-09-15T21:26:14.953699+0000","mapping_epoch":36,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8683520,"num_objects":2120,"num_object_clones":0,"num_object_copies":4240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15246,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16640","reported_seq":23569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.344003+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:15.344003+0000","last_peered":"2024-09-15T21:26:15.344003+0000","last_clean":"2024-09-15T21:26:15.344003+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:15.344003+0000","last_undegraded":"2024-09-15T21:26:15.344003+0000","last_fullsized":"2024-09-15T21:26:15.344003+0000","mapping_epoch":45,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9130008,"num_objects":2230,"num_object_clones":0,"num_object_copies":4460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15363,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16493","reported_seq":16576,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.920362+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:14.920362+0000","last_peered":"2024-09-15T21:26:14.920362+0000","last_clean":"2024-09-15T21:26:14.920362+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:14.920362+0000","last_undegraded":"2024-09-15T21:26:14.920362+0000","last_fullsized":"2024-09-15T21:26:14.920362+0000","mapping_epoch":36,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8507392,"num_objects":2077,"num_object_clones":0,"num_object_copies":4154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16493,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16601","reported_seq":16732,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.585916+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:14.585916+0000","last_peered":"2024-09-15T21:26:14.585916+0000","last_clean":"2024-09-15T21:26:14.585916+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:14.585916+0000","last_undegraded":"2024-09-15T21:26:14.585916+0000","last_fullsized":"2024-09-15T21:26:14.585916+0000","mapping_epoch":35,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8691712,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16317","reported_seq":17142,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.600395+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:14.600395+0000","last_peered":"2024-09-15T21:26:14.600395+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:14.600395+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":2064,"num_object_clones":0,"num_object_copies":4128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13742,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16791","reported_seq":17542,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.593834+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:14.593834+0000","last_peered":"2024-09-15T21:26:14.593834+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:14.593834+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14427,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16278","reported_seq":16411,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.604646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:14.604646+0000","last_peered":"2024-09-15T21:26:14.604646+0000","last_clean":"2024-09-15T21:26:14.604646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:14.604646+0000","last_undegraded":"2024-09-15T21:26:14.604646+0000","last_fullsized":"2024-09-15T21:26:14.604646+0000","mapping_epoch":35,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14063,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16283","reported_seq":23111,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.269957+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:15.269957+0000","last_peered":"2024-09-15T21:26:15.269957+0000","last_clean":"2024-09-15T21:26:15.269957+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:15.269957+0000","last_undegraded":"2024-09-15T21:26:15.269957+0000","last_fullsized":"2024-09-15T21:26:15.269957+0000","mapping_epoch":49,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8568832,"num_objects":2092,"num_object_clones":0,"num_object_copies":4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14869,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16058","reported_seq":16845,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.601566+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:14.601566+0000","last_peered":"2024-09-15T21:26:14.601566+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:14.601566+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8093696,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13584,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1039}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16182","reported_seq":20021,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:15.016496+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:15.016496+0000","last_peered":"2024-09-15T21:26:15.016496+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:15.016496+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8613888,"num_objects":2103,"num_object_clones":0,"num_object_copies":4206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14733,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1857,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16416","reported_seq":16833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.942168+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:14.942168+0000","last_peered":"2024-09-15T21:26:14.942168+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:14.942168+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"45'6400","ondisk_log_start":"45'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8732672,"num_objects":2132,"num_object_clones":0,"num_object_copies":4264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14815,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16593","reported_seq":17341,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.495322+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:14.495322+0000","last_peered":"2024-09-15T21:26:14.495322+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:14.495322+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8671232,"num_objects":2117,"num_object_clones":0,"num_object_copies":4234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13900,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16285","reported_seq":23203,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.373492+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:15.373492+0000","last_peered":"2024-09-15T21:26:15.373492+0000","last_clean":"2024-09-15T21:26:15.373492+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:15.373492+0000","last_undegraded":"2024-09-15T21:26:15.373492+0000","last_fullsized":"2024-09-15T21:26:15.373492+0000","mapping_epoch":51,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16313","reported_seq":16977,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.593883+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:14.593883+0000","last_peered":"2024-09-15T21:26:14.593883+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:14.593883+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8458240,"num_objects":2065,"num_object_clones":0,"num_object_copies":4130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13960,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16573","reported_seq":23672,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.367136+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:15.367136+0000","last_peered":"2024-09-15T21:26:15.367136+0000","last_clean":"2024-09-15T21:26:15.367136+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:15.367136+0000","last_undegraded":"2024-09-15T21:26:15.367136+0000","last_fullsized":"2024-09-15T21:26:15.367136+0000","mapping_epoch":47,"log_start":"45'6500","ondisk_log_start":"45'6500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376937,"num_objects":2046,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14233,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":137978497,"num_objects":33688,"num_object_clones":0,"num_object_copies":67376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234091,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17466,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":441688064,"data_stored":441631214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160877,"ondisk_log_size":160877,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953581,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465928,"kb_used_data":177392,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93905912,"statfs":{"total":96636764160,"available":96159653888,"internally_reserved":0,"allocated":181649408,"data_stored":181348006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149808,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552008,"kb_used_data":155252,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93819832,"statfs":{"total":96636764160,"available":96071507968,"internally_reserved":0,"allocated":158978048,"data_stored":158680718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626527,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516840,"kb_used_data":101976,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93855000,"statfs":{"total":96636764160,"available":96107520000,"internally_reserved":0,"allocated":104423424,"data_stored":104083753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":424799916},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157802496,"data_stored":157799017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102903808,"data_stored":102857988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":180981760,"data_stored":180974209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'16334", 'reported_seq': 17095, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.531847+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:14.531847+0000', 'last_peered': '2024-09-15T21:26:14.531847+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:14.531847+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8921088, 'num_objects': 2178, 'num_object_clones': 0, 'num_object_copies': 4356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16334, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -853}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'16317", 'reported_seq': 17142, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.600395+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:14.600395+0000', 'last_peered': '2024-09-15T21:26:14.600395+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:14.600395+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8454144, 'num_objects': 2064, 'num_object_clones': 0, 'num_object_copies': 4128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13742, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16791", 'reported_seq': 17542, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.593834+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:14.593834+0000', 'last_peered': '2024-09-15T21:26:14.593834+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:14.593834+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8654848, 'num_objects': 2113, 'num_object_clones': 0, 'num_object_copies': 4226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14427, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16058", 'reported_seq': 16845, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.601566+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:14.601566+0000', 'last_peered': '2024-09-15T21:26:14.601566+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:14.601566+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'6000", 'ondisk_log_start': "44'6000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8093696, 'num_objects': 1976, 'num_object_clones': 0, 'num_object_copies': 3952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13584, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1039}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16182", 'reported_seq': 20021, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:15.016496+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:15.016496+0000', 'last_peered': '2024-09-15T21:26:15.016496+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:15.016496+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8613888, 'num_objects': 2103, 'num_object_clones': 0, 'num_object_copies': 4206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1411, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14733, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1857, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 692}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'16416", 'reported_seq': 16833, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.942168+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:14.942168+0000', 'last_peered': '2024-09-15T21:26:14.942168+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:14.942168+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8732672, 'num_objects': 2132, 'num_object_clones': 0, 'num_object_copies': 4264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14815, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1074}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'16593", 'reported_seq': 17341, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.495322+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:14.495322+0000', 'last_peered': '2024-09-15T21:26:14.495322+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:14.495322+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'6500", 'ondisk_log_start': "46'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8671232, 'num_objects': 2117, 'num_object_clones': 0, 'num_object_copies': 4234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13900, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:17.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'16313", 'reported_seq': 16977, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:14.593883+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:14.593883+0000', 'last_peered': '2024-09-15T21:26:14.593883+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:14.593883+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "44'6300", 'ondisk_log_start': "44'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8458240, 'num_objects': 2065, 'num_object_clones': 0, 'num_object_copies': 4130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13960, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1037}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:17.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:17.979 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:17.979 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23099/67380 objects degraded (34.282%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":591,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33690,"data_bytes":137978516,"bytes_used":1571610624,"bytes_avail":288338681856,"bytes_total":289910292480,"degraded_objects":23099,"degraded_total":67380,"degraded_ratio":0.34281685960225589,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":404,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":586},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-15T21:25:26.486200+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:26:18.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:18.064 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:18.862 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:18.864 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:18.945 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-15T21:26:18.523858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":137978516,"num_objects":33690,"num_object_clones":0,"num_object_copies":67380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234093,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17467,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160880,"ondisk_log_size":160880,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534776,"kb_used_data":434620,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281580744,"statfs":{"total":289910292480,"available":288338681856,"internally_reserved":0,"allocated":445050880,"data_stored":444112477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1126485002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,4,6,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19193856,"num_objects":-4686,"num_object_clones":0,"num_object_copies":-9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4686,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013137"},"pg_stats":[{"pgid":"2.f","version":"52'16334","reported_seq":17095,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.531847+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:14.531847+0000","last_peered":"2024-09-15T21:26:14.531847+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:14.531847+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8921088,"num_objects":2178,"num_object_clones":0,"num_object_copies":4356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16520","reported_seq":16630,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.953699+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:14.953699+0000","last_peered":"2024-09-15T21:26:14.953699+0000","last_clean":"2024-09-15T21:26:14.953699+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:14.953699+0000","last_undegraded":"2024-09-15T21:26:14.953699+0000","last_fullsized":"2024-09-15T21:26:14.953699+0000","mapping_epoch":36,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8683520,"num_objects":2120,"num_object_clones":0,"num_object_copies":4240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15246,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16640","reported_seq":23569,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.344003+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:15.344003+0000","last_peered":"2024-09-15T21:26:15.344003+0000","last_clean":"2024-09-15T21:26:15.344003+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:15.344003+0000","last_undegraded":"2024-09-15T21:26:15.344003+0000","last_fullsized":"2024-09-15T21:26:15.344003+0000","mapping_epoch":45,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9130008,"num_objects":2230,"num_object_clones":0,"num_object_copies":4460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15363,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16493","reported_seq":16576,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.920362+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:14.920362+0000","last_peered":"2024-09-15T21:26:14.920362+0000","last_clean":"2024-09-15T21:26:14.920362+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:14.920362+0000","last_undegraded":"2024-09-15T21:26:14.920362+0000","last_fullsized":"2024-09-15T21:26:14.920362+0000","mapping_epoch":36,"log_start":"44'6400","ondisk_log_start":"44'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8507392,"num_objects":2077,"num_object_clones":0,"num_object_copies":4154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16493,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16601","reported_seq":16732,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.585916+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:14.585916+0000","last_peered":"2024-09-15T21:26:14.585916+0000","last_clean":"2024-09-15T21:26:14.585916+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:14.585916+0000","last_undegraded":"2024-09-15T21:26:14.585916+0000","last_fullsized":"2024-09-15T21:26:14.585916+0000","mapping_epoch":35,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8691712,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16317","reported_seq":17142,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.600395+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:14.600395+0000","last_peered":"2024-09-15T21:26:14.600395+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:14.600395+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":2064,"num_object_clones":0,"num_object_copies":4128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13742,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-965}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16791","reported_seq":17542,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.593834+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:14.593834+0000","last_peered":"2024-09-15T21:26:14.593834+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:14.593834+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14427,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16278","reported_seq":16411,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:14.604646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:14.604646+0000","last_peered":"2024-09-15T21:26:14.604646+0000","last_clean":"2024-09-15T21:26:14.604646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:14.604646+0000","last_undegraded":"2024-09-15T21:26:14.604646+0000","last_fullsized":"2024-09-15T21:26:14.604646+0000","mapping_epoch":35,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14063,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16283","reported_seq":23111,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.269957+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:15.269957+0000","last_peered":"2024-09-15T21:26:15.269957+0000","last_clean":"2024-09-15T21:26:15.269957+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:15.269957+0000","last_undegraded":"2024-09-15T21:26:15.269957+0000","last_fullsized":"2024-09-15T21:26:15.269957+0000","mapping_epoch":49,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8568832,"num_objects":2092,"num_object_clones":0,"num_object_copies":4184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14869,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16058","reported_seq":16845,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.601566+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:14.601566+0000","last_peered":"2024-09-15T21:26:14.601566+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:14.601566+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'6000","ondisk_log_start":"44'6000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8093696,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13584,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1039}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16182","reported_seq":20021,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:15.016496+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:15.016496+0000","last_peered":"2024-09-15T21:26:15.016496+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:15.016496+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8613888,"num_objects":2103,"num_object_clones":0,"num_object_copies":4206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14733,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1857,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16416","reported_seq":16833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.942168+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:14.942168+0000","last_peered":"2024-09-15T21:26:14.942168+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:14.942168+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"45'6400","ondisk_log_start":"45'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8732672,"num_objects":2132,"num_object_clones":0,"num_object_copies":4264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14815,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1074}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16593","reported_seq":17341,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.495322+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:14.495322+0000","last_peered":"2024-09-15T21:26:14.495322+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:14.495322+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8671232,"num_objects":2117,"num_object_clones":0,"num_object_copies":4234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13900,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16285","reported_seq":23203,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.373492+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:15.373492+0000","last_peered":"2024-09-15T21:26:15.373492+0000","last_clean":"2024-09-15T21:26:15.373492+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:15.373492+0000","last_undegraded":"2024-09-15T21:26:15.373492+0000","last_fullsized":"2024-09-15T21:26:15.373492+0000","mapping_epoch":51,"log_start":"44'6200","ondisk_log_start":"44'6200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16313","reported_seq":16977,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:14.593883+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:14.593883+0000","last_peered":"2024-09-15T21:26:14.593883+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:14.593883+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"44'6300","ondisk_log_start":"44'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8458240,"num_objects":2065,"num_object_clones":0,"num_object_copies":4130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13960,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16573","reported_seq":23672,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:15.367136+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:15.367136+0000","last_peered":"2024-09-15T21:26:15.367136+0000","last_clean":"2024-09-15T21:26:15.367136+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:15.367136+0000","last_undegraded":"2024-09-15T21:26:15.367136+0000","last_fullsized":"2024-09-15T21:26:15.367136+0000","mapping_epoch":47,"log_start":"45'6500","ondisk_log_start":"45'6500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376937,"num_objects":2046,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14233,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":137978497,"num_objects":33688,"num_object_clones":0,"num_object_copies":67376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234091,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17466,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":441688064,"data_stored":441631214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160877,"ondisk_log_size":160877,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953581,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465928,"kb_used_data":177392,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93905912,"statfs":{"total":96636764160,"available":96159653888,"internally_reserved":0,"allocated":181649408,"data_stored":181348006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149808,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552008,"kb_used_data":155252,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93819832,"statfs":{"total":96636764160,"available":96071507968,"internally_reserved":0,"allocated":158978048,"data_stored":158680718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":406253233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626527,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516840,"kb_used_data":101976,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93855000,"statfs":{"total":96636764160,"available":96107520000,"internally_reserved":0,"allocated":104423424,"data_stored":104083753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":424799916},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157802496,"data_stored":157799017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102903808,"data_stored":102857988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":180981760,"data_stored":180974209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:21.948 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:22.794 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:22.849 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-15T21:26:22.524773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128246420,"num_objects":31314,"num_object_clones":0,"num_object_copies":62628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236469,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17608,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534548,"kb_used_data":416336,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281580972,"statfs":{"total":289910292480,"available":288338915328,"internally_reserved":0,"allocated":426328064,"data_stored":425389661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1144966154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,7,4,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013051"},"pg_stats":[{"pgid":"2.f","version":"52'16477","reported_seq":17239,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.592847+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:19.592847+0000","last_peered":"2024-09-15T21:26:19.592847+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:19.592847+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16672","reported_seq":16782,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.031048+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:20.031048+0000","last_peered":"2024-09-15T21:26:20.031048+0000","last_clean":"2024-09-15T21:26:20.031048+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:20.031048+0000","last_undegraded":"2024-09-15T21:26:20.031048+0000","last_fullsized":"2024-09-15T21:26:20.031048+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8060928,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15398,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16798","reported_seq":23728,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.407025+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:20.407025+0000","last_peered":"2024-09-15T21:26:20.407025+0000","last_clean":"2024-09-15T21:26:20.407025+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:20.407025+0000","last_undegraded":"2024-09-15T21:26:20.407025+0000","last_fullsized":"2024-09-15T21:26:20.407025+0000","mapping_epoch":45,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8482840,"num_objects":2072,"num_object_clones":0,"num_object_copies":4144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15521,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16654","reported_seq":16737,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.997278+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:19.997278+0000","last_peered":"2024-09-15T21:26:19.997278+0000","last_clean":"2024-09-15T21:26:19.997278+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:19.997278+0000","last_undegraded":"2024-09-15T21:26:19.997278+0000","last_fullsized":"2024-09-15T21:26:19.997278+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16654,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16764","reported_seq":16897,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.612362+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:19.612362+0000","last_peered":"2024-09-15T21:26:19.612362+0000","last_clean":"2024-09-15T21:26:19.612362+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:19.612362+0000","last_undegraded":"2024-09-15T21:26:19.612362+0000","last_fullsized":"2024-09-15T21:26:19.612362+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8024064,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14622,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16486","reported_seq":17311,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.653297+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:19.653297+0000","last_peered":"2024-09-15T21:26:19.653297+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:19.653297+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7761920,"num_objects":1895,"num_object_clones":0,"num_object_copies":3790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13911,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16950","reported_seq":17696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.634532+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:19.634532+0000","last_peered":"2024-09-15T21:26:19.634532+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:19.634532+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8003584,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14586,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16404","reported_seq":16538,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.578930+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:19.578930+0000","last_peered":"2024-09-15T21:26:19.578930+0000","last_clean":"2024-09-15T21:26:19.578930+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:19.578930+0000","last_undegraded":"2024-09-15T21:26:19.578930+0000","last_fullsized":"2024-09-15T21:26:19.578930+0000","mapping_epoch":35,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16438","reported_seq":23266,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.384683+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:20.384683+0000","last_peered":"2024-09-15T21:26:20.384683+0000","last_clean":"2024-09-15T21:26:20.384683+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:20.384683+0000","last_undegraded":"2024-09-15T21:26:20.384683+0000","last_fullsized":"2024-09-15T21:26:20.384683+0000","mapping_epoch":49,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15024,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16194","reported_seq":16983,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.642209+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:19.642209+0000","last_peered":"2024-09-15T21:26:19.642209+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:19.642209+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":1840,"num_object_clones":0,"num_object_copies":3680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13720,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16301","reported_seq":20422,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.977339+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:19.977339+0000","last_peered":"2024-09-15T21:26:19.977339+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:19.977339+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"45'6300","ondisk_log_start":"45'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14852,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16569","reported_seq":16986,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.970022+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:19.970022+0000","last_peered":"2024-09-15T21:26:19.970022+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:19.970022+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14968,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16733","reported_seq":17481,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.645488+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:19.645488+0000","last_peered":"2024-09-15T21:26:19.645488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:19.645488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14040,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16443","reported_seq":23361,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.420440+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:20.420440+0000","last_peered":"2024-09-15T21:26:20.420440+0000","last_clean":"2024-09-15T21:26:20.420440+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:20.420440+0000","last_undegraded":"2024-09-15T21:26:20.420440+0000","last_fullsized":"2024-09-15T21:26:20.420440+0000","mapping_epoch":51,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8065024,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16444","reported_seq":17108,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.629595+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:19.629595+0000","last_peered":"2024-09-15T21:26:19.629595+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:19.629595+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"45'6400","ondisk_log_start":"45'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14091,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16726","reported_seq":23825,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.444162+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:20.444162+0000","last_peered":"2024-09-15T21:26:20.444162+0000","last_clean":"2024-09-15T21:26:20.444162+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:20.444162+0000","last_undegraded":"2024-09-15T21:26:20.444162+0000","last_fullsized":"2024-09-15T21:26:20.444162+0000","mapping_epoch":47,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128246401,"num_objects":31312,"num_object_clones":0,"num_object_copies":62624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236467,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17607,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":422154240,"data_stored":422097390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465320,"kb_used_data":172624,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93906520,"statfs":{"total":96636764160,"available":96160276480,"internally_reserved":0,"allocated":176766976,"data_stored":176465574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554996,"kb_used_data":148516,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93816844,"statfs":{"total":96636764160,"available":96068448256,"internally_reserved":0,"allocated":152080384,"data_stored":151783054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514232,"kb_used_data":95196,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93857608,"statfs":{"total":96636764160,"available":96110190592,"internally_reserved":0,"allocated":97480704,"data_stored":97141033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150614016,"data_stored":150610537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95432704,"data_stored":95386884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176107520,"data_stored":176099969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:22.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:23.328 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:23.328 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:23.344 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-15T21:26:22.524773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128246420,"num_objects":31314,"num_object_clones":0,"num_object_copies":62628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236469,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17608,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534548,"kb_used_data":416336,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281580972,"statfs":{"total":289910292480,"available":288338915328,"internally_reserved":0,"allocated":426328064,"data_stored":425389661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1144966154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,7,4,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013051"},"pg_stats":[{"pgid":"2.f","version":"52'16477","reported_seq":17239,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.592847+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:19.592847+0000","last_peered":"2024-09-15T21:26:19.592847+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:19.592847+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16672","reported_seq":16782,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.031048+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:20.031048+0000","last_peered":"2024-09-15T21:26:20.031048+0000","last_clean":"2024-09-15T21:26:20.031048+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:20.031048+0000","last_undegraded":"2024-09-15T21:26:20.031048+0000","last_fullsized":"2024-09-15T21:26:20.031048+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8060928,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15398,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16798","reported_seq":23728,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.407025+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:20.407025+0000","last_peered":"2024-09-15T21:26:20.407025+0000","last_clean":"2024-09-15T21:26:20.407025+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:20.407025+0000","last_undegraded":"2024-09-15T21:26:20.407025+0000","last_fullsized":"2024-09-15T21:26:20.407025+0000","mapping_epoch":45,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8482840,"num_objects":2072,"num_object_clones":0,"num_object_copies":4144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15521,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16654","reported_seq":16737,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.997278+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:19.997278+0000","last_peered":"2024-09-15T21:26:19.997278+0000","last_clean":"2024-09-15T21:26:19.997278+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:19.997278+0000","last_undegraded":"2024-09-15T21:26:19.997278+0000","last_fullsized":"2024-09-15T21:26:19.997278+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16654,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16764","reported_seq":16897,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.612362+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:19.612362+0000","last_peered":"2024-09-15T21:26:19.612362+0000","last_clean":"2024-09-15T21:26:19.612362+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:19.612362+0000","last_undegraded":"2024-09-15T21:26:19.612362+0000","last_fullsized":"2024-09-15T21:26:19.612362+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8024064,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14622,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16486","reported_seq":17311,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.653297+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:19.653297+0000","last_peered":"2024-09-15T21:26:19.653297+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:19.653297+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7761920,"num_objects":1895,"num_object_clones":0,"num_object_copies":3790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13911,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16950","reported_seq":17696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.634532+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:19.634532+0000","last_peered":"2024-09-15T21:26:19.634532+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:19.634532+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8003584,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14586,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16404","reported_seq":16538,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.578930+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:19.578930+0000","last_peered":"2024-09-15T21:26:19.578930+0000","last_clean":"2024-09-15T21:26:19.578930+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:19.578930+0000","last_undegraded":"2024-09-15T21:26:19.578930+0000","last_fullsized":"2024-09-15T21:26:19.578930+0000","mapping_epoch":35,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16438","reported_seq":23266,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.384683+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:20.384683+0000","last_peered":"2024-09-15T21:26:20.384683+0000","last_clean":"2024-09-15T21:26:20.384683+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:20.384683+0000","last_undegraded":"2024-09-15T21:26:20.384683+0000","last_fullsized":"2024-09-15T21:26:20.384683+0000","mapping_epoch":49,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15024,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16194","reported_seq":16983,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.642209+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:19.642209+0000","last_peered":"2024-09-15T21:26:19.642209+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:19.642209+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":1840,"num_object_clones":0,"num_object_copies":3680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13720,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16301","reported_seq":20422,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.977339+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:19.977339+0000","last_peered":"2024-09-15T21:26:19.977339+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:19.977339+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"45'6300","ondisk_log_start":"45'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14852,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16569","reported_seq":16986,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.970022+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:19.970022+0000","last_peered":"2024-09-15T21:26:19.970022+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:19.970022+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14968,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16733","reported_seq":17481,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.645488+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:19.645488+0000","last_peered":"2024-09-15T21:26:19.645488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:19.645488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14040,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16443","reported_seq":23361,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.420440+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:20.420440+0000","last_peered":"2024-09-15T21:26:20.420440+0000","last_clean":"2024-09-15T21:26:20.420440+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:20.420440+0000","last_undegraded":"2024-09-15T21:26:20.420440+0000","last_fullsized":"2024-09-15T21:26:20.420440+0000","mapping_epoch":51,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8065024,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16444","reported_seq":17108,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.629595+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:19.629595+0000","last_peered":"2024-09-15T21:26:19.629595+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:19.629595+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"45'6400","ondisk_log_start":"45'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14091,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16726","reported_seq":23825,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.444162+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:20.444162+0000","last_peered":"2024-09-15T21:26:20.444162+0000","last_clean":"2024-09-15T21:26:20.444162+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:20.444162+0000","last_undegraded":"2024-09-15T21:26:20.444162+0000","last_fullsized":"2024-09-15T21:26:20.444162+0000","mapping_epoch":47,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128246401,"num_objects":31312,"num_object_clones":0,"num_object_copies":62624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236467,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17607,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":422154240,"data_stored":422097390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465320,"kb_used_data":172624,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93906520,"statfs":{"total":96636764160,"available":96160276480,"internally_reserved":0,"allocated":176766976,"data_stored":176465574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554996,"kb_used_data":148516,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93816844,"statfs":{"total":96636764160,"available":96068448256,"internally_reserved":0,"allocated":152080384,"data_stored":151783054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514232,"kb_used_data":95196,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93857608,"statfs":{"total":96636764160,"available":96110190592,"internally_reserved":0,"allocated":97480704,"data_stored":97141033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150614016,"data_stored":150610537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95432704,"data_stored":95386884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176107520,"data_stored":176099969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'16477", 'reported_seq': 17239, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.592847+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:19.592847+0000', 'last_peered': '2024-09-15T21:26:19.592847+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:19.592847+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'6400", 'ondisk_log_start': "46'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8335360, 'num_objects': 2035, 'num_object_clones': 0, 'num_object_copies': 4070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16477, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -996}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'16486", 'reported_seq': 17311, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.653297+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:19.653297+0000', 'last_peered': '2024-09-15T21:26:19.653297+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:19.653297+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'6400", 'ondisk_log_start': "46'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7761920, 'num_objects': 1895, 'num_object_clones': 0, 'num_object_copies': 3790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13911, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'16950", 'reported_seq': 17696, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.634532+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:19.634532+0000', 'last_peered': '2024-09-15T21:26:19.634532+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:19.634532+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'6900", 'ondisk_log_start': "46'6900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8003584, 'num_objects': 1954, 'num_object_clones': 0, 'num_object_copies': 3908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14586, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:23.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16194", 'reported_seq': 16983, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.642209+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:19.642209+0000', 'last_peered': '2024-09-15T21:26:19.642209+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:19.642209+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "44'6100", 'ondisk_log_start': "44'6100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7536640, 'num_objects': 1840, 'num_object_clones': 0, 'num_object_copies': 3680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13720, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1175}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16301", 'reported_seq': 20422, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.977339+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:19.977339+0000', 'last_peered': '2024-09-15T21:26:19.977339+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:19.977339+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "45'6300", 'ondisk_log_start': "45'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8126464, 'num_objects': 1984, 'num_object_clones': 0, 'num_object_copies': 3968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14852, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1998, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 714}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'16569", 'reported_seq': 16986, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.970022+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:19.970022+0000', 'last_peered': '2024-09-15T21:26:19.970022+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:19.970022+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'6500", 'ondisk_log_start': "46'6500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8105984, 'num_objects': 1979, 'num_object_clones': 0, 'num_object_copies': 3958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14968, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1227}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'16733", 'reported_seq': 17481, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.645488+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:19.645488+0000', 'last_peered': '2024-09-15T21:26:19.645488+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:19.645488+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8097792, 'num_objects': 1977, 'num_object_clones': 0, 'num_object_copies': 3954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14040, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1153}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:23.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'16444", 'reported_seq': 17108, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:19.629595+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:19.629595+0000', 'last_peered': '2024-09-15T21:26:19.629595+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:19.629595+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7921664, 'num_objects': 1934, 'num_object_clones': 0, 'num_object_copies': 3868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1934, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14091, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:23.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:24.307 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:24.308 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22958/62628 objects degraded (36.658%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":597,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":31314,"data_bytes":128246420,"bytes_used":1571377152,"bytes_avail":288338915328,"bytes_total":289910292480,"degraded_objects":22958,"degraded_total":62628,"degraded_ratio":0.36657724979242512,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:24.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:24.337 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:24.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:24.993 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:25.009 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-15T21:26:24.525178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128246420,"num_objects":31314,"num_object_clones":0,"num_object_copies":62628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236469,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17608,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534548,"kb_used_data":416336,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281580972,"statfs":{"total":289910292480,"available":288338915328,"internally_reserved":0,"allocated":426328064,"data_stored":425389661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1144966154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,7,4,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-296,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013001"},"pg_stats":[{"pgid":"2.f","version":"52'16477","reported_seq":17239,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.592847+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:19.592847+0000","last_peered":"2024-09-15T21:26:19.592847+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:19.592847+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-996}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16672","reported_seq":16782,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.031048+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:20.031048+0000","last_peered":"2024-09-15T21:26:20.031048+0000","last_clean":"2024-09-15T21:26:20.031048+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:20.031048+0000","last_undegraded":"2024-09-15T21:26:20.031048+0000","last_fullsized":"2024-09-15T21:26:20.031048+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8060928,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15398,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16798","reported_seq":23728,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.407025+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:20.407025+0000","last_peered":"2024-09-15T21:26:20.407025+0000","last_clean":"2024-09-15T21:26:20.407025+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:20.407025+0000","last_undegraded":"2024-09-15T21:26:20.407025+0000","last_fullsized":"2024-09-15T21:26:20.407025+0000","mapping_epoch":45,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8482840,"num_objects":2072,"num_object_clones":0,"num_object_copies":4144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15521,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16654","reported_seq":16737,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.997278+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:19.997278+0000","last_peered":"2024-09-15T21:26:19.997278+0000","last_clean":"2024-09-15T21:26:19.997278+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:19.997278+0000","last_undegraded":"2024-09-15T21:26:19.997278+0000","last_fullsized":"2024-09-15T21:26:19.997278+0000","mapping_epoch":36,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16654,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16764","reported_seq":16897,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.612362+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:19.612362+0000","last_peered":"2024-09-15T21:26:19.612362+0000","last_clean":"2024-09-15T21:26:19.612362+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:19.612362+0000","last_undegraded":"2024-09-15T21:26:19.612362+0000","last_fullsized":"2024-09-15T21:26:19.612362+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8024064,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14622,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16486","reported_seq":17311,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.653297+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:19.653297+0000","last_peered":"2024-09-15T21:26:19.653297+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:19.653297+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7761920,"num_objects":1895,"num_object_clones":0,"num_object_copies":3790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13911,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'16950","reported_seq":17696,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.634532+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:19.634532+0000","last_peered":"2024-09-15T21:26:19.634532+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:19.634532+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8003584,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14586,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16404","reported_seq":16538,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:19.578930+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:19.578930+0000","last_peered":"2024-09-15T21:26:19.578930+0000","last_clean":"2024-09-15T21:26:19.578930+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:19.578930+0000","last_undegraded":"2024-09-15T21:26:19.578930+0000","last_fullsized":"2024-09-15T21:26:19.578930+0000","mapping_epoch":35,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16438","reported_seq":23266,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.384683+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:20.384683+0000","last_peered":"2024-09-15T21:26:20.384683+0000","last_clean":"2024-09-15T21:26:20.384683+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:20.384683+0000","last_undegraded":"2024-09-15T21:26:20.384683+0000","last_fullsized":"2024-09-15T21:26:20.384683+0000","mapping_epoch":49,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15024,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16194","reported_seq":16983,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.642209+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:19.642209+0000","last_peered":"2024-09-15T21:26:19.642209+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:19.642209+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"44'6100","ondisk_log_start":"44'6100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":1840,"num_object_clones":0,"num_object_copies":3680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13720,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1175}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16301","reported_seq":20422,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.977339+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:19.977339+0000","last_peered":"2024-09-15T21:26:19.977339+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:19.977339+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"45'6300","ondisk_log_start":"45'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8126464,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14852,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16569","reported_seq":16986,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.970022+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:19.970022+0000","last_peered":"2024-09-15T21:26:19.970022+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:19.970022+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8105984,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14968,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16733","reported_seq":17481,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.645488+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:19.645488+0000","last_peered":"2024-09-15T21:26:19.645488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:19.645488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14040,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16443","reported_seq":23361,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.420440+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:20.420440+0000","last_peered":"2024-09-15T21:26:20.420440+0000","last_clean":"2024-09-15T21:26:20.420440+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:20.420440+0000","last_undegraded":"2024-09-15T21:26:20.420440+0000","last_fullsized":"2024-09-15T21:26:20.420440+0000","mapping_epoch":51,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8065024,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14028,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16444","reported_seq":17108,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:19.629595+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:19.629595+0000","last_peered":"2024-09-15T21:26:19.629595+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:19.629595+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"45'6400","ondisk_log_start":"45'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7921664,"num_objects":1934,"num_object_clones":0,"num_object_copies":3868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14091,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16726","reported_seq":23825,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:20.444162+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:20.444162+0000","last_peered":"2024-09-15T21:26:20.444162+0000","last_clean":"2024-09-15T21:26:20.444162+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:20.444162+0000","last_undegraded":"2024-09-15T21:26:20.444162+0000","last_fullsized":"2024-09-15T21:26:20.444162+0000","mapping_epoch":47,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128246401,"num_objects":31312,"num_object_clones":0,"num_object_copies":62624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236467,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17607,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":422154240,"data_stored":422097390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953582,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465320,"kb_used_data":172624,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93906520,"statfs":{"total":96636764160,"available":96160276480,"internally_reserved":0,"allocated":176766976,"data_stored":176465574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554996,"kb_used_data":148516,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93816844,"statfs":{"total":96636764160,"available":96068448256,"internally_reserved":0,"allocated":152080384,"data_stored":151783054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626528,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514232,"kb_used_data":95196,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93857608,"statfs":{"total":96636764160,"available":96110190592,"internally_reserved":0,"allocated":97480704,"data_stored":97141033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":429059756},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150614016,"data_stored":150610537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95432704,"data_stored":95386884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176107520,"data_stored":176099969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:28.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:28.854 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:28.856 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:28.936 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-15T21:26:28.526123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":118633108,"num_objects":28967,"num_object_clones":0,"num_object_copies":57934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238816,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17762,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497496,"kb_used_data":395936,"kb_used_omap":12,"kb_used_meta":1101491,"kb_avail":281618024,"statfs":{"total":289910292480,"available":288376856576,"internally_reserved":0,"allocated":405438464,"data_stored":404500061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1127926794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,6,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19345408,"num_objects":-4723,"num_object_clones":0,"num_object_copies":-9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013195"},"pg_stats":[{"pgid":"2.f","version":"52'16634","reported_seq":17395,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.708324+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:24.708324+0000","last_peered":"2024-09-15T21:26:24.708324+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:24.708324+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7692288,"num_objects":1878,"num_object_clones":0,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16634,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1153}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16818","reported_seq":16928,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.888662+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:24.888662+0000","last_peered":"2024-09-15T21:26:24.888662+0000","last_clean":"2024-09-15T21:26:24.888662+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:24.888662+0000","last_undegraded":"2024-09-15T21:26:24.888662+0000","last_fullsized":"2024-09-15T21:26:24.888662+0000","mapping_epoch":36,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7462912,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15544,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16944","reported_seq":23874,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.478047+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:25.478047+0000","last_peered":"2024-09-15T21:26:25.478047+0000","last_clean":"2024-09-15T21:26:25.478047+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:25.478047+0000","last_undegraded":"2024-09-15T21:26:25.478047+0000","last_fullsized":"2024-09-15T21:26:25.478047+0000","mapping_epoch":45,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884824,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15667,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16790","reported_seq":16873,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.884081+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:24.884081+0000","last_peered":"2024-09-15T21:26:24.884081+0000","last_clean":"2024-09-15T21:26:24.884081+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:24.884081+0000","last_undegraded":"2024-09-15T21:26:24.884081+0000","last_fullsized":"2024-09-15T21:26:24.884081+0000","mapping_epoch":36,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290880,"num_objects":1780,"num_object_clones":0,"num_object_copies":3560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16790,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16926","reported_seq":17059,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.586673+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:24.586673+0000","last_peered":"2024-09-15T21:26:24.586673+0000","last_clean":"2024-09-15T21:26:24.586673+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:24.586673+0000","last_undegraded":"2024-09-15T21:26:24.586673+0000","last_fullsized":"2024-09-15T21:26:24.586673+0000","mapping_epoch":35,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7360512,"num_objects":1797,"num_object_clones":0,"num_object_copies":3594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14784,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16630","reported_seq":17458,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.638691+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:24.638691+0000","last_peered":"2024-09-15T21:26:24.638691+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:24.638691+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7172096,"num_objects":1751,"num_object_clones":0,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14055,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17105","reported_seq":17856,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.667530+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:24.667530+0000","last_peered":"2024-09-15T21:26:24.667530+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:24.667530+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7368704,"num_objects":1799,"num_object_clones":0,"num_object_copies":3598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14741,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16545","reported_seq":16679,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.576159+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:24.576159+0000","last_peered":"2024-09-15T21:26:24.576159+0000","last_clean":"2024-09-15T21:26:24.576159+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:24.576159+0000","last_undegraded":"2024-09-15T21:26:24.576159+0000","last_fullsized":"2024-09-15T21:26:24.576159+0000","mapping_epoch":35,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614464,"num_objects":1859,"num_object_clones":0,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14330,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16587","reported_seq":23415,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.477792+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:25.477792+0000","last_peered":"2024-09-15T21:26:25.477792+0000","last_clean":"2024-09-15T21:26:25.477792+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:25.477792+0000","last_undegraded":"2024-09-15T21:26:25.477792+0000","last_fullsized":"2024-09-15T21:26:25.477792+0000","mapping_epoch":49,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7323648,"num_objects":1788,"num_object_clones":0,"num_object_copies":3576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15173,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16329","reported_seq":17118,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.615443+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:24.615443+0000","last_peered":"2024-09-15T21:26:24.615443+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:24.615443+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"45'6300","ondisk_log_start":"45'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6983680,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13855,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16449","reported_seq":20878,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:25.049986+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:25.049986+0000","last_peered":"2024-09-15T21:26:25.049986+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:25.049986+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7520256,"num_objects":1836,"num_object_clones":0,"num_object_copies":3672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16724","reported_seq":17141,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.905822+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:24.905822+0000","last_peered":"2024-09-15T21:26:24.905822+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:24.905822+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":1824,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15123,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16868","reported_seq":17616,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.693818+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:24.693818+0000","last_peered":"2024-09-15T21:26:24.693818+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:24.693818+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7544832,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16580","reported_seq":23498,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.504627+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:25.504627+0000","last_peered":"2024-09-15T21:26:25.504627+0000","last_clean":"2024-09-15T21:26:25.504627+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:25.504627+0000","last_undegraded":"2024-09-15T21:26:25.504627+0000","last_fullsized":"2024-09-15T21:26:25.504627+0000","mapping_epoch":51,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7503872,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16609","reported_seq":17273,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.634165+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:24.634165+0000","last_peered":"2024-09-15T21:26:24.634165+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:24.634165+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7245824,"num_objects":1769,"num_object_clones":0,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16862","reported_seq":23961,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.520704+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:25.520704+0000","last_peered":"2024-09-15T21:26:25.520704+0000","last_clean":"2024-09-15T21:26:25.520704+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:25.520704+0000","last_undegraded":"2024-09-15T21:26:25.520704+0000","last_fullsized":"2024-09-15T21:26:25.520704+0000","mapping_epoch":47,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193193,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14522,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":118633089,"num_objects":28965,"num_object_clones":0,"num_object_copies":57930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238814,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17761,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":402948096,"data_stored":402891246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448028,"kb_used_data":167820,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93923812,"statfs":{"total":96636764160,"available":96177983488,"internally_reserved":0,"allocated":171847680,"data_stored":171546278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551916,"kb_used_data":141276,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93819924,"statfs":{"total":96636764160,"available":96071602176,"internally_reserved":0,"allocated":144666624,"data_stored":144369294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":420474545},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497552,"kb_used_data":86840,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93874288,"statfs":{"total":96636764160,"available":96127270912,"internally_reserved":0,"allocated":88924160,"data_stored":88584489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143265792,"data_stored":143262313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88272896,"data_stored":88227076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171409408,"data_stored":171401857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:28.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:29.386 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:29.387 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:29.455 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-15T21:26:28.526123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":118633108,"num_objects":28967,"num_object_clones":0,"num_object_copies":57934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28967,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238816,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17762,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160703,"ondisk_log_size":160703,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497496,"kb_used_data":395936,"kb_used_omap":12,"kb_used_meta":1101491,"kb_avail":281618024,"statfs":{"total":289910292480,"available":288376856576,"internally_reserved":0,"allocated":405438464,"data_stored":404500061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1127926794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,6,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19345408,"num_objects":-4723,"num_object_clones":0,"num_object_copies":-9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013195"},"pg_stats":[{"pgid":"2.f","version":"52'16634","reported_seq":17395,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.708324+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:24.708324+0000","last_peered":"2024-09-15T21:26:24.708324+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:24.708324+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7692288,"num_objects":1878,"num_object_clones":0,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16634,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1153}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16818","reported_seq":16928,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.888662+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:24.888662+0000","last_peered":"2024-09-15T21:26:24.888662+0000","last_clean":"2024-09-15T21:26:24.888662+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:24.888662+0000","last_undegraded":"2024-09-15T21:26:24.888662+0000","last_fullsized":"2024-09-15T21:26:24.888662+0000","mapping_epoch":36,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7462912,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15544,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16944","reported_seq":23874,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.478047+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:25.478047+0000","last_peered":"2024-09-15T21:26:25.478047+0000","last_clean":"2024-09-15T21:26:25.478047+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:25.478047+0000","last_undegraded":"2024-09-15T21:26:25.478047+0000","last_fullsized":"2024-09-15T21:26:25.478047+0000","mapping_epoch":45,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884824,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15667,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16790","reported_seq":16873,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.884081+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:24.884081+0000","last_peered":"2024-09-15T21:26:24.884081+0000","last_clean":"2024-09-15T21:26:24.884081+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:24.884081+0000","last_undegraded":"2024-09-15T21:26:24.884081+0000","last_fullsized":"2024-09-15T21:26:24.884081+0000","mapping_epoch":36,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290880,"num_objects":1780,"num_object_clones":0,"num_object_copies":3560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16790,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'16926","reported_seq":17059,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.586673+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:24.586673+0000","last_peered":"2024-09-15T21:26:24.586673+0000","last_clean":"2024-09-15T21:26:24.586673+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:24.586673+0000","last_undegraded":"2024-09-15T21:26:24.586673+0000","last_fullsized":"2024-09-15T21:26:24.586673+0000","mapping_epoch":35,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7360512,"num_objects":1797,"num_object_clones":0,"num_object_copies":3594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14784,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16630","reported_seq":17458,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.638691+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:24.638691+0000","last_peered":"2024-09-15T21:26:24.638691+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:24.638691+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7172096,"num_objects":1751,"num_object_clones":0,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14055,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17105","reported_seq":17856,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.667530+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:24.667530+0000","last_peered":"2024-09-15T21:26:24.667530+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:24.667530+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7368704,"num_objects":1799,"num_object_clones":0,"num_object_copies":3598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14741,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16545","reported_seq":16679,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:24.576159+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:24.576159+0000","last_peered":"2024-09-15T21:26:24.576159+0000","last_clean":"2024-09-15T21:26:24.576159+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:24.576159+0000","last_undegraded":"2024-09-15T21:26:24.576159+0000","last_fullsized":"2024-09-15T21:26:24.576159+0000","mapping_epoch":35,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614464,"num_objects":1859,"num_object_clones":0,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14330,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16587","reported_seq":23415,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.477792+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:25.477792+0000","last_peered":"2024-09-15T21:26:25.477792+0000","last_clean":"2024-09-15T21:26:25.477792+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:25.477792+0000","last_undegraded":"2024-09-15T21:26:25.477792+0000","last_fullsized":"2024-09-15T21:26:25.477792+0000","mapping_epoch":49,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7323648,"num_objects":1788,"num_object_clones":0,"num_object_copies":3576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15173,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16329","reported_seq":17118,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.615443+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:24.615443+0000","last_peered":"2024-09-15T21:26:24.615443+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:24.615443+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"45'6300","ondisk_log_start":"45'6300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6983680,"num_objects":1705,"num_object_clones":0,"num_object_copies":3410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13855,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16449","reported_seq":20878,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:25.049986+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:25.049986+0000","last_peered":"2024-09-15T21:26:25.049986+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:25.049986+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7520256,"num_objects":1836,"num_object_clones":0,"num_object_copies":3672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16724","reported_seq":17141,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.905822+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:24.905822+0000","last_peered":"2024-09-15T21:26:24.905822+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:24.905822+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":1824,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15123,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1382}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'16868","reported_seq":17616,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.693818+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:24.693818+0000","last_peered":"2024-09-15T21:26:24.693818+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:24.693818+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7544832,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16580","reported_seq":23498,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.504627+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:25.504627+0000","last_peered":"2024-09-15T21:26:25.504627+0000","last_clean":"2024-09-15T21:26:25.504627+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:25.504627+0000","last_undegraded":"2024-09-15T21:26:25.504627+0000","last_fullsized":"2024-09-15T21:26:25.504627+0000","mapping_epoch":51,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7503872,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16609","reported_seq":17273,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:24.634165+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:24.634165+0000","last_peered":"2024-09-15T21:26:24.634165+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:24.634165+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7245824,"num_objects":1769,"num_object_clones":0,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16862","reported_seq":23961,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.520704+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:25.520704+0000","last_peered":"2024-09-15T21:26:25.520704+0000","last_clean":"2024-09-15T21:26:25.520704+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:25.520704+0000","last_undegraded":"2024-09-15T21:26:25.520704+0000","last_fullsized":"2024-09-15T21:26:25.520704+0000","mapping_epoch":47,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193193,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14522,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":118633089,"num_objects":28965,"num_object_clones":0,"num_object_copies":57930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22804,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238814,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17761,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":402948096,"data_stored":402891246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160700,"ondisk_log_size":160700,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448028,"kb_used_data":167820,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93923812,"statfs":{"total":96636764160,"available":96177983488,"internally_reserved":0,"allocated":171847680,"data_stored":171546278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551916,"kb_used_data":141276,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93819924,"statfs":{"total":96636764160,"available":96071602176,"internally_reserved":0,"allocated":144666624,"data_stored":144369294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":420474545},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,3,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497552,"kb_used_data":86840,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93874288,"statfs":{"total":96636764160,"available":96127270912,"internally_reserved":0,"allocated":88924160,"data_stored":88584489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143265792,"data_stored":143262313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88272896,"data_stored":88227076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171409408,"data_stored":171401857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'16634", 'reported_seq': 17395, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.708324+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:24.708324+0000', 'last_peered': '2024-09-15T21:26:24.708324+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:24.708324+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'6600", 'ondisk_log_start': "46'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7692288, 'num_objects': 1878, 'num_object_clones': 0, 'num_object_copies': 3756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16634, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1153}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'16630", 'reported_seq': 17458, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.638691+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:24.638691+0000', 'last_peered': '2024-09-15T21:26:24.638691+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:24.638691+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'6600", 'ondisk_log_start': "46'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7172096, 'num_objects': 1751, 'num_object_clones': 0, 'num_object_copies': 3502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14055, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:29.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'17105", 'reported_seq': 17856, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.667530+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:24.667530+0000', 'last_peered': '2024-09-15T21:26:24.667530+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:24.667530+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'7100", 'ondisk_log_start': "46'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7368704, 'num_objects': 1799, 'num_object_clones': 0, 'num_object_copies': 3598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14741, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16329", 'reported_seq': 17118, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.615443+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:24.615443+0000', 'last_peered': '2024-09-15T21:26:24.615443+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:24.615443+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "45'6300", 'ondisk_log_start': "45'6300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6983680, 'num_objects': 1705, 'num_object_clones': 0, 'num_object_copies': 3410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13855, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1310}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16449", 'reported_seq': 20878, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:25.049986+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:25.049986+0000', 'last_peered': '2024-09-15T21:26:25.049986+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:25.049986+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "46'6400", 'ondisk_log_start': "46'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7520256, 'num_objects': 1836, 'num_object_clones': 0, 'num_object_copies': 3672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1836, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15000, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 720}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'16724", 'reported_seq': 17141, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.905822+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:24.905822+0000', 'last_peered': '2024-09-15T21:26:24.905822+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:24.905822+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7471104, 'num_objects': 1824, 'num_object_clones': 0, 'num_object_copies': 3648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15123, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1382}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'16868", 'reported_seq': 17616, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.693818+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:24.693818+0000', 'last_peered': '2024-09-15T21:26:24.693818+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:24.693818+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'6800", 'ondisk_log_start': "46'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7544832, 'num_objects': 1842, 'num_object_clones': 0, 'num_object_copies': 3684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14175, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:29.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'16609", 'reported_seq': 17273, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:24.634165+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:24.634165+0000', 'last_peered': '2024-09-15T21:26:24.634165+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:24.634165+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'6600", 'ondisk_log_start': "46'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7245824, 'num_objects': 1769, 'num_object_clones': 0, 'num_object_copies': 3538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14256, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:29.460 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:30.379 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:30.379 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22804/57934 objects degraded (39.362%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":603,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28967,"data_bytes":118633108,"bytes_used":1533435904,"bytes_avail":288376856576,"bytes_total":289910292480,"degraded_objects":22804,"degraded_total":57934,"degraded_ratio":0.39362032657852031,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:30.429 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:30.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:31.391 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:31.407 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-15T21:26:30.526509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":111317652,"num_objects":27181,"num_object_clones":0,"num_object_copies":54362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":240602,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17903,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498204,"kb_used_data":384160,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281617316,"statfs":{"total":289910292480,"available":288376131584,"internally_reserved":0,"allocated":393379840,"data_stored":392441437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1140706314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26660864,"num_objects":-6509,"num_object_clones":0,"num_object_copies":-13018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"2.f","version":"52'16795","reported_seq":17557,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.732793+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:29.732793+0000","last_peered":"2024-09-15T21:26:29.732793+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:29.732793+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1314}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16958","reported_seq":17068,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.026201+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:30.026201+0000","last_peered":"2024-09-15T21:26:30.026201+0000","last_clean":"2024-09-15T21:26:30.026201+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:30.026201+0000","last_undegraded":"2024-09-15T21:26:30.026201+0000","last_fullsized":"2024-09-15T21:26:30.026201+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6889472,"num_objects":1682,"num_object_clones":0,"num_object_copies":3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15684,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'16944","reported_seq":23874,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.478047+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:25.478047+0000","last_peered":"2024-09-15T21:26:25.478047+0000","last_clean":"2024-09-15T21:26:25.478047+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:25.478047+0000","last_undegraded":"2024-09-15T21:26:25.478047+0000","last_fullsized":"2024-09-15T21:26:25.478047+0000","mapping_epoch":45,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884824,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15667,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16936","reported_seq":17019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.925973+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:29.925973+0000","last_peered":"2024-09-15T21:26:29.925973+0000","last_clean":"2024-09-15T21:26:29.925973+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:29.925973+0000","last_undegraded":"2024-09-15T21:26:29.925973+0000","last_fullsized":"2024-09-15T21:26:29.925973+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17066","reported_seq":17199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.703324+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:29.703324+0000","last_peered":"2024-09-15T21:26:29.703324+0000","last_clean":"2024-09-15T21:26:29.703324+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:29.703324+0000","last_undegraded":"2024-09-15T21:26:29.703324+0000","last_fullsized":"2024-09-15T21:26:29.703324+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6787072,"num_objects":1657,"num_object_clones":0,"num_object_copies":3314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14924,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16778","reported_seq":17606,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.641851+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:29.641851+0000","last_peered":"2024-09-15T21:26:29.641851+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:29.641851+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14203,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17253","reported_seq":18004,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.700096+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:29.700096+0000","last_peered":"2024-09-15T21:26:29.700096+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:29.700096+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6762496,"num_objects":1651,"num_object_clones":0,"num_object_copies":3302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14889,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16707","reported_seq":16841,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.712705+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:29.712705+0000","last_peered":"2024-09-15T21:26:29.712705+0000","last_clean":"2024-09-15T21:26:29.712705+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:29.712705+0000","last_undegraded":"2024-09-15T21:26:29.712705+0000","last_fullsized":"2024-09-15T21:26:29.712705+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14492,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16587","reported_seq":23415,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.477792+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:25.477792+0000","last_peered":"2024-09-15T21:26:25.477792+0000","last_clean":"2024-09-15T21:26:25.477792+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:25.477792+0000","last_undegraded":"2024-09-15T21:26:25.477792+0000","last_fullsized":"2024-09-15T21:26:25.477792+0000","mapping_epoch":49,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7323648,"num_objects":1788,"num_object_clones":0,"num_object_copies":3576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15173,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16471","reported_seq":17260,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.722858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:29.722858+0000","last_peered":"2024-09-15T21:26:29.722858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:29.722858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6402048,"num_objects":1563,"num_object_clones":0,"num_object_copies":3126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1452}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16610","reported_seq":21321,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:30.064637+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:30.064637+0000","last_peered":"2024-09-15T21:26:30.064637+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:30.064637+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16875","reported_seq":17292,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.988485+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:29.988485+0000","last_peered":"2024-09-15T21:26:29.988485+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:29.988485+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6852608,"num_objects":1673,"num_object_clones":0,"num_object_copies":3346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15274,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1533}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17015","reported_seq":17763,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.654882+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:29.654882+0000","last_peered":"2024-09-15T21:26:29.654882+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:29.654882+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16580","reported_seq":23498,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.504627+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:25.504627+0000","last_peered":"2024-09-15T21:26:25.504627+0000","last_clean":"2024-09-15T21:26:25.504627+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:25.504627+0000","last_undegraded":"2024-09-15T21:26:25.504627+0000","last_fullsized":"2024-09-15T21:26:25.504627+0000","mapping_epoch":51,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7503872,"num_objects":1832,"num_object_clones":0,"num_object_copies":3664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16749","reported_seq":17412,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.709306+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:29.709306+0000","last_peered":"2024-09-15T21:26:29.709306+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:29.709306+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14396,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16862","reported_seq":23961,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:25.520704+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:25.520704+0000","last_peered":"2024-09-15T21:26:25.520704+0000","last_clean":"2024-09-15T21:26:25.520704+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:25.520704+0000","last_undegraded":"2024-09-15T21:26:25.520704+0000","last_fullsized":"2024-09-15T21:26:25.520704+0000","mapping_epoch":47,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193193,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14522,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":111317633,"num_objects":27179,"num_object_clones":0,"num_object_copies":54358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27179,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":240600,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17902,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":390762496,"data_stored":390705646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447412,"kb_used_data":163040,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93924428,"statfs":{"total":96636764160,"available":96178614272,"internally_reserved":0,"allocated":166952960,"data_stored":166651558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553240,"kb_used_data":134280,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93818600,"statfs":{"total":96636764160,"available":96070246400,"internally_reserved":0,"allocated":137502720,"data_stored":137205390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497552,"kb_used_data":86840,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93874288,"statfs":{"total":96636764160,"available":96127270912,"internally_reserved":0,"allocated":88924160,"data_stored":88584489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":420540076},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136069120,"data_stored":136065641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88272896,"data_stored":88227076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166420480,"data_stored":166412929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:34.410 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:35.124 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:35.127 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:35.179 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-15T21:26:34.527276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108778132,"num_objects":26561,"num_object_clones":0,"num_object_copies":53122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26561,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241222,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17903,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517212,"kb_used_data":376804,"kb_used_omap":12,"kb_used_meta":1140339,"kb_avail":281598308,"statfs":{"total":289910292480,"available":288356667392,"internally_reserved":0,"allocated":385847296,"data_stored":384908893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1167707146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19468288,"num_objects":-4753,"num_object_clones":0,"num_object_copies":-9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002502"},"pg_stats":[{"pgid":"2.f","version":"52'16795","reported_seq":17557,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.732793+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:29.732793+0000","last_peered":"2024-09-15T21:26:29.732793+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:29.732793+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1314}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16958","reported_seq":17068,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.026201+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:30.026201+0000","last_peered":"2024-09-15T21:26:30.026201+0000","last_clean":"2024-09-15T21:26:30.026201+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:30.026201+0000","last_undegraded":"2024-09-15T21:26:30.026201+0000","last_fullsized":"2024-09-15T21:26:30.026201+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6889472,"num_objects":1682,"num_object_clones":0,"num_object_copies":3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15684,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17108","reported_seq":24038,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.545718+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:30.545718+0000","last_peered":"2024-09-15T21:26:30.545718+0000","last_clean":"2024-09-15T21:26:30.545718+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:30.545718+0000","last_undegraded":"2024-09-15T21:26:30.545718+0000","last_fullsized":"2024-09-15T21:26:30.545718+0000","mapping_epoch":45,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7213080,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15831,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16936","reported_seq":17019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.925973+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:29.925973+0000","last_peered":"2024-09-15T21:26:29.925973+0000","last_clean":"2024-09-15T21:26:29.925973+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:29.925973+0000","last_undegraded":"2024-09-15T21:26:29.925973+0000","last_fullsized":"2024-09-15T21:26:29.925973+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17066","reported_seq":17199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.703324+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:29.703324+0000","last_peered":"2024-09-15T21:26:29.703324+0000","last_clean":"2024-09-15T21:26:29.703324+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:29.703324+0000","last_undegraded":"2024-09-15T21:26:29.703324+0000","last_fullsized":"2024-09-15T21:26:29.703324+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6787072,"num_objects":1657,"num_object_clones":0,"num_object_copies":3314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14924,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16778","reported_seq":17606,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.641851+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:29.641851+0000","last_peered":"2024-09-15T21:26:29.641851+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:29.641851+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14203,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17253","reported_seq":18004,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.700096+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:29.700096+0000","last_peered":"2024-09-15T21:26:29.700096+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:29.700096+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6762496,"num_objects":1651,"num_object_clones":0,"num_object_copies":3302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14889,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16707","reported_seq":16841,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.712705+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:29.712705+0000","last_peered":"2024-09-15T21:26:29.712705+0000","last_clean":"2024-09-15T21:26:29.712705+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:29.712705+0000","last_undegraded":"2024-09-15T21:26:29.712705+0000","last_fullsized":"2024-09-15T21:26:29.712705+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14492,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16735","reported_seq":23563,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.420161+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:30.420161+0000","last_peered":"2024-09-15T21:26:30.420161+0000","last_clean":"2024-09-15T21:26:30.420161+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:30.420161+0000","last_undegraded":"2024-09-15T21:26:30.420161+0000","last_fullsized":"2024-09-15T21:26:30.420161+0000","mapping_epoch":49,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6717440,"num_objects":1640,"num_object_clones":0,"num_object_copies":3280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15321,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16471","reported_seq":17260,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.722858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:29.722858+0000","last_peered":"2024-09-15T21:26:29.722858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:29.722858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6402048,"num_objects":1563,"num_object_clones":0,"num_object_copies":3126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1452}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16610","reported_seq":21321,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:30.064637+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:30.064637+0000","last_peered":"2024-09-15T21:26:30.064637+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:30.064637+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16875","reported_seq":17292,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.988485+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:29.988485+0000","last_peered":"2024-09-15T21:26:29.988485+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:29.988485+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6852608,"num_objects":1673,"num_object_clones":0,"num_object_copies":3346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15274,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1533}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17015","reported_seq":17763,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.654882+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:29.654882+0000","last_peered":"2024-09-15T21:26:29.654882+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:29.654882+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16753","reported_seq":23671,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.600848+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:30.600848+0000","last_peered":"2024-09-15T21:26:30.600848+0000","last_clean":"2024-09-15T21:26:30.600848+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:30.600848+0000","last_undegraded":"2024-09-15T21:26:30.600848+0000","last_fullsized":"2024-09-15T21:26:30.600848+0000","mapping_epoch":51,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14338,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16749","reported_seq":17412,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.709306+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:29.709306+0000","last_peered":"2024-09-15T21:26:29.709306+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:29.709306+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14396,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16997","reported_seq":24096,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.604496+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:30.604496+0000","last_peered":"2024-09-15T21:26:30.604496+0000","last_clean":"2024-09-15T21:26:30.604496+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:30.604496+0000","last_undegraded":"2024-09-15T21:26:30.604496+0000","last_fullsized":"2024-09-15T21:26:30.604496+0000","mapping_epoch":47,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6640233,"num_objects":1622,"num_object_clones":0,"num_object_copies":3244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108778113,"num_objects":26559,"num_object_clones":0,"num_object_copies":53118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241220,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17902,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":383275008,"data_stored":383218158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447412,"kb_used_data":163040,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93924428,"statfs":{"total":96636764160,"available":96178614272,"internally_reserved":0,"allocated":166952960,"data_stored":166651558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553240,"kb_used_data":134280,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93818600,"statfs":{"total":96636764160,"available":96070246400,"internally_reserved":0,"allocated":137502720,"data_stored":137205390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516560,"kb_used_data":79484,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93855280,"statfs":{"total":96636764160,"available":96107806720,"internally_reserved":0,"allocated":81391616,"data_stored":81051945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136069120,"data_stored":136065641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80785408,"data_stored":80739588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166420480,"data_stored":166412929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:35.181 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:35.626 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:35.627 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:35.663 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-15T21:26:34.527276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108778132,"num_objects":26561,"num_object_clones":0,"num_object_copies":53122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26561,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241222,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17903,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517212,"kb_used_data":376804,"kb_used_omap":12,"kb_used_meta":1140339,"kb_avail":281598308,"statfs":{"total":289910292480,"available":288356667392,"internally_reserved":0,"allocated":385847296,"data_stored":384908893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1167707146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,8,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19468288,"num_objects":-4753,"num_object_clones":0,"num_object_copies":-9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4753,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002502"},"pg_stats":[{"pgid":"2.f","version":"52'16795","reported_seq":17557,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.732793+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:29.732793+0000","last_peered":"2024-09-15T21:26:29.732793+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:29.732793+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1314}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'16958","reported_seq":17068,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.026201+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:30.026201+0000","last_peered":"2024-09-15T21:26:30.026201+0000","last_clean":"2024-09-15T21:26:30.026201+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:30.026201+0000","last_undegraded":"2024-09-15T21:26:30.026201+0000","last_fullsized":"2024-09-15T21:26:30.026201+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6889472,"num_objects":1682,"num_object_clones":0,"num_object_copies":3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15684,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17108","reported_seq":24038,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.545718+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:30.545718+0000","last_peered":"2024-09-15T21:26:30.545718+0000","last_clean":"2024-09-15T21:26:30.545718+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:30.545718+0000","last_undegraded":"2024-09-15T21:26:30.545718+0000","last_fullsized":"2024-09-15T21:26:30.545718+0000","mapping_epoch":45,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7213080,"num_objects":1762,"num_object_clones":0,"num_object_copies":3524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15831,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'16936","reported_seq":17019,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.925973+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:29.925973+0000","last_peered":"2024-09-15T21:26:29.925973+0000","last_clean":"2024-09-15T21:26:29.925973+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:29.925973+0000","last_undegraded":"2024-09-15T21:26:29.925973+0000","last_fullsized":"2024-09-15T21:26:29.925973+0000","mapping_epoch":36,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6692864,"num_objects":1634,"num_object_clones":0,"num_object_copies":3268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17066","reported_seq":17199,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.703324+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:29.703324+0000","last_peered":"2024-09-15T21:26:29.703324+0000","last_clean":"2024-09-15T21:26:29.703324+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:29.703324+0000","last_undegraded":"2024-09-15T21:26:29.703324+0000","last_fullsized":"2024-09-15T21:26:29.703324+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6787072,"num_objects":1657,"num_object_clones":0,"num_object_copies":3314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14924,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16778","reported_seq":17606,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.641851+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:29.641851+0000","last_peered":"2024-09-15T21:26:29.641851+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:29.641851+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6565888,"num_objects":1603,"num_object_clones":0,"num_object_copies":3206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14203,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17253","reported_seq":18004,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.700096+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:29.700096+0000","last_peered":"2024-09-15T21:26:29.700096+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:29.700096+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6762496,"num_objects":1651,"num_object_clones":0,"num_object_copies":3302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14889,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16707","reported_seq":16841,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:29.712705+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:29.712705+0000","last_peered":"2024-09-15T21:26:29.712705+0000","last_clean":"2024-09-15T21:26:29.712705+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:29.712705+0000","last_undegraded":"2024-09-15T21:26:29.712705+0000","last_fullsized":"2024-09-15T21:26:29.712705+0000","mapping_epoch":35,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6950912,"num_objects":1697,"num_object_clones":0,"num_object_copies":3394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14492,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16735","reported_seq":23563,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.420161+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:30.420161+0000","last_peered":"2024-09-15T21:26:30.420161+0000","last_clean":"2024-09-15T21:26:30.420161+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:30.420161+0000","last_undegraded":"2024-09-15T21:26:30.420161+0000","last_fullsized":"2024-09-15T21:26:30.420161+0000","mapping_epoch":49,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6717440,"num_objects":1640,"num_object_clones":0,"num_object_copies":3280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15321,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16471","reported_seq":17260,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.722858+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:29.722858+0000","last_peered":"2024-09-15T21:26:29.722858+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:29.722858+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6400","ondisk_log_start":"46'6400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6402048,"num_objects":1563,"num_object_clones":0,"num_object_copies":3126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13997,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1452}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16610","reported_seq":21321,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:30.064637+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:30.064637+0000","last_peered":"2024-09-15T21:26:30.064637+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:30.064637+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6600","ondisk_log_start":"46'6600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'16875","reported_seq":17292,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.988485+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:29.988485+0000","last_peered":"2024-09-15T21:26:29.988485+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:29.988485+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6852608,"num_objects":1673,"num_object_clones":0,"num_object_copies":3346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15274,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1533}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17015","reported_seq":17763,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.654882+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:29.654882+0000","last_peered":"2024-09-15T21:26:29.654882+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:29.654882+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16753","reported_seq":23671,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.600848+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:30.600848+0000","last_peered":"2024-09-15T21:26:30.600848+0000","last_clean":"2024-09-15T21:26:30.600848+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:30.600848+0000","last_undegraded":"2024-09-15T21:26:30.600848+0000","last_fullsized":"2024-09-15T21:26:30.600848+0000","mapping_epoch":51,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14338,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16749","reported_seq":17412,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:29.709306+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:29.709306+0000","last_peered":"2024-09-15T21:26:29.709306+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:29.709306+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672384,"num_objects":1629,"num_object_clones":0,"num_object_copies":3258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14396,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1473}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'16997","reported_seq":24096,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:30.604496+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:30.604496+0000","last_peered":"2024-09-15T21:26:30.604496+0000","last_clean":"2024-09-15T21:26:30.604496+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:30.604496+0000","last_undegraded":"2024-09-15T21:26:30.604496+0000","last_fullsized":"2024-09-15T21:26:30.604496+0000","mapping_epoch":47,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6640233,"num_objects":1622,"num_object_clones":0,"num_object_copies":3244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108778113,"num_objects":26559,"num_object_clones":0,"num_object_copies":53118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241220,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17902,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":383275008,"data_stored":383218158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953584,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447412,"kb_used_data":163040,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93924428,"statfs":{"total":96636764160,"available":96178614272,"internally_reserved":0,"allocated":166952960,"data_stored":166651558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149811,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553240,"kb_used_data":134280,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93818600,"statfs":{"total":96636764160,"available":96070246400,"internally_reserved":0,"allocated":137502720,"data_stored":137205390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516560,"kb_used_data":79484,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93855280,"statfs":{"total":96636764160,"available":96107806720,"internally_reserved":0,"allocated":81391616,"data_stored":81051945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136069120,"data_stored":136065641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80785408,"data_stored":80739588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166420480,"data_stored":166412929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'16795", 'reported_seq': 17557, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.732793+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:29.732793+0000', 'last_peered': '2024-09-15T21:26:29.732793+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:29.732793+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7032832, 'num_objects': 1717, 'num_object_clones': 0, 'num_object_copies': 3434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16795, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1314}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'16778", 'reported_seq': 17606, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.641851+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:29.641851+0000', 'last_peered': '2024-09-15T21:26:29.641851+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:29.641851+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6565888, 'num_objects': 1603, 'num_object_clones': 0, 'num_object_copies': 3206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1603, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14203, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'17253", 'reported_seq': 18004, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.700096+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:29.700096+0000', 'last_peered': '2024-09-15T21:26:29.700096+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:29.700096+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'7200", 'ondisk_log_start': "46'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6762496, 'num_objects': 1651, 'num_object_clones': 0, 'num_object_copies': 3302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14889, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16471", 'reported_seq': 17260, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.722858+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:29.722858+0000', 'last_peered': '2024-09-15T21:26:29.722858+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:29.722858+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'6400", 'ondisk_log_start': "46'6400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6402048, 'num_objects': 1563, 'num_object_clones': 0, 'num_object_copies': 3126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13997, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1452}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16610", 'reported_seq': 21321, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:30.064637+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:30.064637+0000', 'last_peered': '2024-09-15T21:26:30.064637+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:30.064637+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "46'6600", 'ondisk_log_start': "46'6600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6860800, 'num_objects': 1675, 'num_object_clones': 0, 'num_object_copies': 3350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 975, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15161, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2293, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 700}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'16875", 'reported_seq': 17292, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.988485+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:29.988485+0000', 'last_peered': '2024-09-15T21:26:29.988485+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:29.988485+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'6800", 'ondisk_log_start': "46'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6852608, 'num_objects': 1673, 'num_object_clones': 0, 'num_object_copies': 3346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15274, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1533}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'17015", 'reported_seq': 17763, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.654882+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:29.654882+0000', 'last_peered': '2024-09-15T21:26:29.654882+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:29.654882+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6942720, 'num_objects': 1695, 'num_object_clones': 0, 'num_object_copies': 3390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14322, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:35.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'16749", 'reported_seq': 17412, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:29.709306+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:29.709306+0000', 'last_peered': '2024-09-15T21:26:29.709306+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:29.709306+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6672384, 'num_objects': 1629, 'num_object_clones': 0, 'num_object_copies': 3258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14396, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:35.667 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:36.793 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:36.793 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22663/53122 objects degraded (42.662%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26561,"data_bytes":108778132,"bytes_used":1553625088,"bytes_avail":288356667392,"bytes_total":289910292480,"degraded_objects":22663,"degraded_total":53122,"degraded_ratio":0.42662173863935848,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:36.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:36.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:37.775 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:37.848 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-15T21:26:36.527693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99308180,"num_objects":24249,"num_object_clones":0,"num_object_copies":48498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243534,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482476,"kb_used_data":358696,"kb_used_omap":12,"kb_used_meta":1123699,"kb_avail":281633044,"statfs":{"total":289910292480,"available":288392237056,"internally_reserved":0,"allocated":367304704,"data_stored":366366301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1150667786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,4,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28938240,"num_objects":-7065,"num_object_clones":0,"num_object_copies":-14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7065,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002515"},"pg_stats":[{"pgid":"2.f","version":"52'16931","reported_seq":17693,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.804347+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:34.804347+0000","last_peered":"2024-09-15T21:26:34.804347+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:34.804347+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6475776,"num_objects":1581,"num_object_clones":0,"num_object_copies":3162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16931,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17100","reported_seq":17210,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.042560+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:35.042560+0000","last_peered":"2024-09-15T21:26:35.042560+0000","last_clean":"2024-09-15T21:26:35.042560+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:35.042560+0000","last_undegraded":"2024-09-15T21:26:35.042560+0000","last_fullsized":"2024-09-15T21:26:35.042560+0000","mapping_epoch":36,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6307840,"num_objects":1540,"num_object_clones":0,"num_object_copies":3080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17261","reported_seq":24191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.546946+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:35.546946+0000","last_peered":"2024-09-15T21:26:35.546946+0000","last_clean":"2024-09-15T21:26:35.546946+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:35.546946+0000","last_undegraded":"2024-09-15T21:26:35.546946+0000","last_fullsized":"2024-09-15T21:26:35.546946+0000","mapping_epoch":45,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586392,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15984,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17058","reported_seq":17141,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.939153+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:34.939153+0000","last_peered":"2024-09-15T21:26:34.939153+0000","last_clean":"2024-09-15T21:26:34.939153+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:34.939153+0000","last_undegraded":"2024-09-15T21:26:34.939153+0000","last_fullsized":"2024-09-15T21:26:34.939153+0000","mapping_epoch":36,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6193152,"num_objects":1512,"num_object_clones":0,"num_object_copies":3024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17058,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17232","reported_seq":17365,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.763980+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:34.763980+0000","last_peered":"2024-09-15T21:26:34.763980+0000","last_clean":"2024-09-15T21:26:34.763980+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:34.763980+0000","last_undegraded":"2024-09-15T21:26:34.763980+0000","last_fullsized":"2024-09-15T21:26:34.763980+0000","mapping_epoch":35,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15090,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'16908","reported_seq":17736,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.723359+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:34.723359+0000","last_peered":"2024-09-15T21:26:34.723359+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:34.723359+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6033408,"num_objects":1473,"num_object_clones":0,"num_object_copies":2946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14333,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17405","reported_seq":18156,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.692171+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:34.692171+0000","last_peered":"2024-09-15T21:26:34.692171+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:34.692171+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6139904,"num_objects":1499,"num_object_clones":0,"num_object_copies":2998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15041,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'16865","reported_seq":16996,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.757613+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:34.757613+0000","last_peered":"2024-09-15T21:26:34.757613+0000","last_clean":"2024-09-15T21:26:34.757613+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:34.757613+0000","last_undegraded":"2024-09-15T21:26:34.757613+0000","last_fullsized":"2024-09-15T21:26:34.757613+0000","mapping_epoch":35,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6303744,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14650,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16864","reported_seq":23692,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.563250+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:35.563250+0000","last_peered":"2024-09-15T21:26:35.563250+0000","last_clean":"2024-09-15T21:26:35.563250+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:35.563250+0000","last_undegraded":"2024-09-15T21:26:35.563250+0000","last_fullsized":"2024-09-15T21:26:35.563250+0000","mapping_epoch":49,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15450,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16590","reported_seq":17379,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.769865+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:34.769865+0000","last_peered":"2024-09-15T21:26:34.769865+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:34.769865+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6500","ondisk_log_start":"46'6500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5914624,"num_objects":1444,"num_object_clones":0,"num_object_copies":2888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14116,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1571}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16746","reported_seq":21789,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:35.065419+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:35.065419+0000","last_peered":"2024-09-15T21:26:35.065419+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:35.065419+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6303744,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15297,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2459,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17048","reported_seq":17463,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:35.038714+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:35.038714+0000","last_peered":"2024-09-15T21:26:35.038714+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:35.038714+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6144000,"num_objects":1500,"num_object_clones":0,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17181","reported_seq":17929,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.803498+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:34.803498+0000","last_peered":"2024-09-15T21:26:34.803498+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:34.803498+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14488,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1601}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16911","reported_seq":23829,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.617402+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:35.617402+0000","last_peered":"2024-09-15T21:26:35.617402+0000","last_clean":"2024-09-15T21:26:35.617402+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:35.617402+0000","last_undegraded":"2024-09-15T21:26:35.617402+0000","last_fullsized":"2024-09-15T21:26:35.617402+0000","mapping_epoch":51,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6148096,"num_objects":1501,"num_object_clones":0,"num_object_copies":3002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'16898","reported_seq":17562,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:34.669010+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:34.669010+0000","last_peered":"2024-09-15T21:26:34.669010+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:34.669010+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6062080,"num_objects":1480,"num_object_clones":0,"num_object_copies":2960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14545,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17120","reported_seq":24219,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.646828+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:35.646828+0000","last_peered":"2024-09-15T21:26:35.646828+0000","last_clean":"2024-09-15T21:26:35.646828+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:35.646828+0000","last_undegraded":"2024-09-15T21:26:35.646828+0000","last_fullsized":"2024-09-15T21:26:35.646828+0000","mapping_epoch":47,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6136425,"num_objects":1499,"num_object_clones":0,"num_object_copies":2998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14780,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":99308161,"num_objects":24247,"num_object_clones":0,"num_object_copies":48494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243532,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364195840,"data_stored":364138990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953585,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446840,"kb_used_data":158312,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93925000,"statfs":{"total":96636764160,"available":96179200000,"internally_reserved":0,"allocated":162111488,"data_stored":161810086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149812,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534184,"kb_used_data":127700,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93837656,"statfs":{"total":96636764160,"available":96089759744,"internally_reserved":0,"allocated":130764800,"data_stored":130467470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":72684,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":74428416,"data_stored":74088745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":439021228},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128897024,"data_stored":128893545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73744384,"data_stored":73698564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161554432,"data_stored":161546881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:40.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:41.649 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:41.701 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-15T21:26:40.528582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92078740,"num_objects":22484,"num_object_clones":0,"num_object_copies":44968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245299,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18244,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483068,"kb_used_data":346808,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281632452,"statfs":{"total":289910292480,"available":288391630848,"internally_reserved":0,"allocated":355131392,"data_stored":354192989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1163447306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,10,13,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26554368,"num_objects":-6483,"num_object_clones":0,"num_object_copies":-12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6483,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002458"},"pg_stats":[{"pgid":"2.f","version":"52'17070","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.843526+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:39.843526+0000","last_peered":"2024-09-15T21:26:39.843526+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:39.843526+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":1442,"num_object_clones":0,"num_object_copies":2884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17248","reported_seq":17358,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.036107+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:40.036107+0000","last_peered":"2024-09-15T21:26:40.036107+0000","last_clean":"2024-09-15T21:26:40.036107+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:40.036107+0000","last_undegraded":"2024-09-15T21:26:40.036107+0000","last_fullsized":"2024-09-15T21:26:40.036107+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":1392,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15974,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17261","reported_seq":24191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.546946+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:35.546946+0000","last_peered":"2024-09-15T21:26:35.546946+0000","last_clean":"2024-09-15T21:26:35.546946+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:35.546946+0000","last_undegraded":"2024-09-15T21:26:35.546946+0000","last_fullsized":"2024-09-15T21:26:35.546946+0000","mapping_epoch":45,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586392,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15984,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17205","reported_seq":17288,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.060237+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:40.060237+0000","last_peered":"2024-09-15T21:26:40.060237+0000","last_clean":"2024-09-15T21:26:40.060237+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:40.060237+0000","last_undegraded":"2024-09-15T21:26:40.060237+0000","last_fullsized":"2024-09-15T21:26:40.060237+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5591040,"num_objects":1365,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17381","reported_seq":17514,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.668307+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:39.668307+0000","last_peered":"2024-09-15T21:26:39.668307+0000","last_clean":"2024-09-15T21:26:39.668307+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:39.668307+0000","last_undegraded":"2024-09-15T21:26:39.668307+0000","last_fullsized":"2024-09-15T21:26:39.668307+0000","mapping_epoch":35,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5496832,"num_objects":1342,"num_object_clones":0,"num_object_copies":2684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17046","reported_seq":17873,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.810097+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:39.810097+0000","last_peered":"2024-09-15T21:26:39.810097+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:39.810097+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14471,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17552","reported_seq":18303,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.700218+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:39.700218+0000","last_peered":"2024-09-15T21:26:39.700218+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:39.700218+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17016","reported_seq":17150,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.666765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:39.666765+0000","last_peered":"2024-09-15T21:26:39.666765+0000","last_clean":"2024-09-15T21:26:39.666765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:39.666765+0000","last_undegraded":"2024-09-15T21:26:39.666765+0000","last_fullsized":"2024-09-15T21:26:39.666765+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5685248,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14801,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16864","reported_seq":23692,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.563250+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:35.563250+0000","last_peered":"2024-09-15T21:26:35.563250+0000","last_clean":"2024-09-15T21:26:35.563250+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:35.563250+0000","last_undegraded":"2024-09-15T21:26:35.563250+0000","last_fullsized":"2024-09-15T21:26:35.563250+0000","mapping_epoch":49,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15450,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16744","reported_seq":17533,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.652630+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:39.652630+0000","last_peered":"2024-09-15T21:26:39.652630+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:39.652630+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16877","reported_seq":22270,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.078754+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:40.078754+0000","last_peered":"2024-09-15T21:26:40.078754+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:40.078754+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":1408,"num_object_clones":0,"num_object_copies":2816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17213","reported_seq":17630,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.000666+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:40.000666+0000","last_peered":"2024-09-15T21:26:40.000666+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:40.000666+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15612,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1871}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17330","reported_seq":18078,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.673339+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:39.673339+0000","last_peered":"2024-09-15T21:26:39.673339+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:39.673339+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5652480,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14637,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16911","reported_seq":23829,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.617402+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:35.617402+0000","last_peered":"2024-09-15T21:26:35.617402+0000","last_clean":"2024-09-15T21:26:35.617402+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:35.617402+0000","last_undegraded":"2024-09-15T21:26:35.617402+0000","last_fullsized":"2024-09-15T21:26:35.617402+0000","mapping_epoch":51,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6148096,"num_objects":1501,"num_object_clones":0,"num_object_copies":3002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17045","reported_seq":17709,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.824702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:39.824702+0000","last_peered":"2024-09-15T21:26:39.824702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:39.824702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5459968,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14692,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17120","reported_seq":24219,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.646828+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:35.646828+0000","last_peered":"2024-09-15T21:26:35.646828+0000","last_clean":"2024-09-15T21:26:35.646828+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:35.646828+0000","last_undegraded":"2024-09-15T21:26:35.646828+0000","last_fullsized":"2024-09-15T21:26:35.646828+0000","mapping_epoch":47,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6136425,"num_objects":1499,"num_object_clones":0,"num_object_copies":2998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14780,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92078721,"num_objects":22482,"num_object_clones":0,"num_object_copies":44964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245297,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18243,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352186368,"data_stored":352129518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953586,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446372,"kb_used_data":153684,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93925468,"statfs":{"total":96636764160,"available":96179679232,"internally_reserved":0,"allocated":157372416,"data_stored":157071014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149813,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535244,"kb_used_data":120440,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93836596,"statfs":{"total":96636764160,"available":96088674304,"internally_reserved":0,"allocated":123330560,"data_stored":123033230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":72684,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":74428416,"data_stored":74088745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":439021228},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121683968,"data_stored":121680489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73744384,"data_stored":73698564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156758016,"data_stored":156750465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:41.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:42.296 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:42.297 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:42.366 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-15T21:26:40.528582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92078740,"num_objects":22484,"num_object_clones":0,"num_object_copies":44968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245299,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18244,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483068,"kb_used_data":346808,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281632452,"statfs":{"total":289910292480,"available":288391630848,"internally_reserved":0,"allocated":355131392,"data_stored":354192989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1163447306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,10,13,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26554368,"num_objects":-6483,"num_object_clones":0,"num_object_copies":-12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6483,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002458"},"pg_stats":[{"pgid":"2.f","version":"52'17070","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.843526+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:39.843526+0000","last_peered":"2024-09-15T21:26:39.843526+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:39.843526+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":1442,"num_object_clones":0,"num_object_copies":2884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17248","reported_seq":17358,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.036107+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:40.036107+0000","last_peered":"2024-09-15T21:26:40.036107+0000","last_clean":"2024-09-15T21:26:40.036107+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:40.036107+0000","last_undegraded":"2024-09-15T21:26:40.036107+0000","last_fullsized":"2024-09-15T21:26:40.036107+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":1392,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15974,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17261","reported_seq":24191,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.546946+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:35.546946+0000","last_peered":"2024-09-15T21:26:35.546946+0000","last_clean":"2024-09-15T21:26:35.546946+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:35.546946+0000","last_undegraded":"2024-09-15T21:26:35.546946+0000","last_fullsized":"2024-09-15T21:26:35.546946+0000","mapping_epoch":45,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6586392,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15984,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17205","reported_seq":17288,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.060237+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:40.060237+0000","last_peered":"2024-09-15T21:26:40.060237+0000","last_clean":"2024-09-15T21:26:40.060237+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:40.060237+0000","last_undegraded":"2024-09-15T21:26:40.060237+0000","last_fullsized":"2024-09-15T21:26:40.060237+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5591040,"num_objects":1365,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17381","reported_seq":17514,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.668307+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:39.668307+0000","last_peered":"2024-09-15T21:26:39.668307+0000","last_clean":"2024-09-15T21:26:39.668307+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:39.668307+0000","last_undegraded":"2024-09-15T21:26:39.668307+0000","last_fullsized":"2024-09-15T21:26:39.668307+0000","mapping_epoch":35,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5496832,"num_objects":1342,"num_object_clones":0,"num_object_copies":2684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17046","reported_seq":17873,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.810097+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:39.810097+0000","last_peered":"2024-09-15T21:26:39.810097+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:39.810097+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14471,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17552","reported_seq":18303,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.700218+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:39.700218+0000","last_peered":"2024-09-15T21:26:39.700218+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:39.700218+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17016","reported_seq":17150,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.666765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:39.666765+0000","last_peered":"2024-09-15T21:26:39.666765+0000","last_clean":"2024-09-15T21:26:39.666765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:39.666765+0000","last_undegraded":"2024-09-15T21:26:39.666765+0000","last_fullsized":"2024-09-15T21:26:39.666765+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5685248,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14801,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724869+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724869+0000","last_peered":"2024-09-15T21:25:03.724869+0000","last_clean":"2024-09-15T21:25:03.724869+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724869+0000","last_undegraded":"2024-09-15T21:25:03.724869+0000","last_fullsized":"2024-09-15T21:25:03.724869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'16864","reported_seq":23692,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.563250+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:35.563250+0000","last_peered":"2024-09-15T21:26:35.563250+0000","last_clean":"2024-09-15T21:26:35.563250+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:35.563250+0000","last_undegraded":"2024-09-15T21:26:35.563250+0000","last_fullsized":"2024-09-15T21:26:35.563250+0000","mapping_epoch":49,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15450,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16744","reported_seq":17533,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.652630+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:39.652630+0000","last_peered":"2024-09-15T21:26:39.652630+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:39.652630+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16877","reported_seq":22270,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.078754+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:40.078754+0000","last_peered":"2024-09-15T21:26:40.078754+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:40.078754+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":1408,"num_object_clones":0,"num_object_copies":2816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17213","reported_seq":17630,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.000666+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:40.000666+0000","last_peered":"2024-09-15T21:26:40.000666+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:40.000666+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15612,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1871}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17330","reported_seq":18078,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.673339+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:39.673339+0000","last_peered":"2024-09-15T21:26:39.673339+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:39.673339+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5652480,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14637,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'16911","reported_seq":23829,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.617402+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:35.617402+0000","last_peered":"2024-09-15T21:26:35.617402+0000","last_clean":"2024-09-15T21:26:35.617402+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:35.617402+0000","last_undegraded":"2024-09-15T21:26:35.617402+0000","last_fullsized":"2024-09-15T21:26:35.617402+0000","mapping_epoch":51,"log_start":"46'6900","ondisk_log_start":"46'6900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6148096,"num_objects":1501,"num_object_clones":0,"num_object_copies":3002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17045","reported_seq":17709,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.824702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:39.824702+0000","last_peered":"2024-09-15T21:26:39.824702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:39.824702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5459968,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14692,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17120","reported_seq":24219,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:35.646828+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:35.646828+0000","last_peered":"2024-09-15T21:26:35.646828+0000","last_clean":"2024-09-15T21:26:35.646828+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:35.646828+0000","last_undegraded":"2024-09-15T21:26:35.646828+0000","last_fullsized":"2024-09-15T21:26:35.646828+0000","mapping_epoch":47,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6136425,"num_objects":1499,"num_object_clones":0,"num_object_copies":2998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14780,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92078721,"num_objects":22482,"num_object_clones":0,"num_object_copies":44964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245297,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18243,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352186368,"data_stored":352129518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953586,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446372,"kb_used_data":153684,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93925468,"statfs":{"total":96636764160,"available":96179679232,"internally_reserved":0,"allocated":157372416,"data_stored":157071014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149813,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535244,"kb_used_data":120440,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93836596,"statfs":{"total":96636764160,"available":96088674304,"internally_reserved":0,"allocated":123330560,"data_stored":123033230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626531,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":72684,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":74428416,"data_stored":74088745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":439021228},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121683968,"data_stored":121680489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73744384,"data_stored":73698564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156758016,"data_stored":156750465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:42.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'17070", 'reported_seq': 17833, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.843526+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:39.843526+0000', 'last_peered': '2024-09-15T21:26:39.843526+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:39.843526+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5906432, 'num_objects': 1442, 'num_object_clones': 0, 'num_object_copies': 2884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17070, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1589}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'17046", 'reported_seq': 17873, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.810097+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:39.810097+0000', 'last_peered': '2024-09-15T21:26:39.810097+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:39.810097+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5468160, 'num_objects': 1335, 'num_object_clones': 0, 'num_object_copies': 2670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14471, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1694}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'17552", 'reported_seq': 18303, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.700218+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:39.700218+0000', 'last_peered': '2024-09-15T21:26:39.700218+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:39.700218+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'7500", 'ondisk_log_start': "46'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5537792, 'num_objects': 1352, 'num_object_clones': 0, 'num_object_copies': 2704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15188, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1812}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16744", 'reported_seq': 17533, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.652630+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:39.652630+0000', 'last_peered': '2024-09-15T21:26:39.652630+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:39.652630+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'6700", 'ondisk_log_start': "46'6700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5283840, 'num_objects': 1290, 'num_object_clones': 0, 'num_object_copies': 2580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14270, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1725}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:42.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'16877", 'reported_seq': 22270, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:40.078754+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:40.078754+0000', 'last_peered': '2024-09-15T21:26:40.078754+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:40.078754+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "46'6800", 'ondisk_log_start': "46'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5767168, 'num_objects': 1408, 'num_object_clones': 0, 'num_object_copies': 2816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 634, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15428, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2634, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 774}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'17213", 'reported_seq': 17630, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:40.000666+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:40.000666+0000', 'last_peered': '2024-09-15T21:26:40.000666+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:40.000666+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'7200", 'ondisk_log_start': "46'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5468160, 'num_objects': 1335, 'num_object_clones': 0, 'num_object_copies': 2670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15612, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1871}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'17330", 'reported_seq': 18078, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.673339+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:39.673339+0000', 'last_peered': '2024-09-15T21:26:39.673339+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:39.673339+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'7300", 'ondisk_log_start': "46'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5652480, 'num_objects': 1380, 'num_object_clones': 0, 'num_object_copies': 2760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14637, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1750}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:42.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'17045", 'reported_seq': 17709, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.824702+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:39.824702+0000', 'last_peered': '2024-09-15T21:26:39.824702+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:39.824702+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5459968, 'num_objects': 1333, 'num_object_clones': 0, 'num_object_copies': 2666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14692, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.369 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:43.274 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:43.274 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22322/43682 objects degraded (51.101%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":616,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21841,"data_bytes":89445012,"bytes_used":1515450368,"bytes_avail":288394842112,"bytes_total":289910292480,"degraded_objects":22322,"degraded_total":43682,"degraded_ratio":0.51101140057689665,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:43.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:43.295 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:43.825 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:43.825 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:43.877 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-15T21:26:42.529032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89445012,"num_objects":21841,"num_object_clones":0,"num_object_copies":43682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21841,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245942,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18244,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479932,"kb_used_data":339520,"kb_used_omap":12,"kb_used_meta":1140339,"kb_avail":281635588,"statfs":{"total":289910292480,"available":288394842112,"internally_reserved":0,"allocated":347668480,"data_stored":346730077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1167707146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,10,12,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21872640,"num_objects":-5340,"num_object_clones":0,"num_object_copies":-10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5340,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002523"},"pg_stats":[{"pgid":"2.f","version":"52'17070","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.843526+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:39.843526+0000","last_peered":"2024-09-15T21:26:39.843526+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:39.843526+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":1442,"num_object_clones":0,"num_object_copies":2884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17248","reported_seq":17358,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.036107+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:40.036107+0000","last_peered":"2024-09-15T21:26:40.036107+0000","last_clean":"2024-09-15T21:26:40.036107+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:40.036107+0000","last_undegraded":"2024-09-15T21:26:40.036107+0000","last_fullsized":"2024-09-15T21:26:40.036107+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5701632,"num_objects":1392,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15974,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17435","reported_seq":24363,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.699729+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:40.699729+0000","last_peered":"2024-09-15T21:26:40.699729+0000","last_clean":"2024-09-15T21:26:40.699729+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:40.699729+0000","last_undegraded":"2024-09-15T21:26:40.699729+0000","last_fullsized":"2024-09-15T21:26:40.699729+0000","mapping_epoch":45,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5873688,"num_objects":1435,"num_object_clones":0,"num_object_copies":2870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16158,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17205","reported_seq":17288,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.060237+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:40.060237+0000","last_peered":"2024-09-15T21:26:40.060237+0000","last_clean":"2024-09-15T21:26:40.060237+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:40.060237+0000","last_undegraded":"2024-09-15T21:26:40.060237+0000","last_fullsized":"2024-09-15T21:26:40.060237+0000","mapping_epoch":36,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5591040,"num_objects":1365,"num_object_clones":0,"num_object_copies":2730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17381","reported_seq":17514,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.668307+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:39.668307+0000","last_peered":"2024-09-15T21:26:39.668307+0000","last_clean":"2024-09-15T21:26:39.668307+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:39.668307+0000","last_undegraded":"2024-09-15T21:26:39.668307+0000","last_fullsized":"2024-09-15T21:26:39.668307+0000","mapping_epoch":35,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5496832,"num_objects":1342,"num_object_clones":0,"num_object_copies":2684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17046","reported_seq":17873,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.810097+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:39.810097+0000","last_peered":"2024-09-15T21:26:39.810097+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:39.810097+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14471,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17552","reported_seq":18303,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.700218+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:39.700218+0000","last_peered":"2024-09-15T21:26:39.700218+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:39.700218+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5537792,"num_objects":1352,"num_object_clones":0,"num_object_copies":2704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15188,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1812}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17016","reported_seq":17150,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:39.666765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:39.666765+0000","last_peered":"2024-09-15T21:26:39.666765+0000","last_clean":"2024-09-15T21:26:39.666765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:39.666765+0000","last_undegraded":"2024-09-15T21:26:39.666765+0000","last_fullsized":"2024-09-15T21:26:39.666765+0000","mapping_epoch":35,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5685248,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14801,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17006","reported_seq":23834,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.739416+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:40.739416+0000","last_peered":"2024-09-15T21:26:40.739416+0000","last_clean":"2024-09-15T21:26:40.739416+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:40.739416+0000","last_undegraded":"2024-09-15T21:26:40.739416+0000","last_fullsized":"2024-09-15T21:26:40.739416+0000","mapping_epoch":49,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5607424,"num_objects":1369,"num_object_clones":0,"num_object_copies":2738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15592,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16744","reported_seq":17533,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.652630+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:39.652630+0000","last_peered":"2024-09-15T21:26:39.652630+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:39.652630+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6700","ondisk_log_start":"46'6700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14270,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'16877","reported_seq":22270,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.078754+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:40.078754+0000","last_peered":"2024-09-15T21:26:40.078754+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:40.078754+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5767168,"num_objects":1408,"num_object_clones":0,"num_object_copies":2816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":634,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2634,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17213","reported_seq":17630,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:40.000666+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:40.000666+0000","last_peered":"2024-09-15T21:26:40.000666+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:40.000666+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468160,"num_objects":1335,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15612,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1871}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17330","reported_seq":18078,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.673339+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:39.673339+0000","last_peered":"2024-09-15T21:26:39.673339+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:39.673339+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5652480,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14637,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17072","reported_seq":23990,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.723333+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:40.723333+0000","last_peered":"2024-09-15T21:26:40.723333+0000","last_clean":"2024-09-15T21:26:40.723333+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:40.723333+0000","last_undegraded":"2024-09-15T21:26:40.723333+0000","last_fullsized":"2024-09-15T21:26:40.723333+0000","mapping_epoch":51,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":1340,"num_object_clones":0,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17045","reported_seq":17709,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.824702+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:39.824702+0000","last_peered":"2024-09-15T21:26:39.824702+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:39.824702+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5459968,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14692,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17286","reported_seq":24385,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:40.777432+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:40.777432+0000","last_peered":"2024-09-15T21:26:40.777432+0000","last_clean":"2024-09-15T21:26:40.777432+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:40.777432+0000","last_undegraded":"2024-09-15T21:26:40.777432+0000","last_fullsized":"2024-09-15T21:26:40.777432+0000","mapping_epoch":47,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5456489,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14946,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":89444993,"num_objects":21839,"num_object_clones":0,"num_object_copies":43678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245940,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18243,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344567808,"data_stored":344510958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953586,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":446372,"kb_used_data":153684,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93925468,"statfs":{"total":96636764160,"available":96179679232,"internally_reserved":0,"allocated":157372416,"data_stored":157071014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149813,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535244,"kb_used_data":120440,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93836596,"statfs":{"total":96636764160,"available":96088674304,"internally_reserved":0,"allocated":123330560,"data_stored":123033230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498316,"kb_used_data":65396,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93873524,"statfs":{"total":96636764160,"available":96126488576,"internally_reserved":0,"allocated":66965504,"data_stored":66625833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":443281068},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121683968,"data_stored":121680489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66125824,"data_stored":66080004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156758016,"data_stored":156750465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:46.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:47.618 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:47.619 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:47.667 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-15T21:26:46.529898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79737492,"num_objects":19471,"num_object_clones":0,"num_object_copies":38942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248312,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18395,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1448820,"kb_used_data":320896,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281666700,"statfs":{"total":289910292480,"available":288426700800,"internally_reserved":0,"allocated":328597504,"data_stored":327659101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1154927626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,4,9,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29040640,"num_objects":-7090,"num_object_clones":0,"num_object_copies":-14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002622"},"pg_stats":[{"pgid":"2.f","version":"52'17238","reported_seq":17998,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.956183+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:44.956183+0000","last_peered":"2024-09-15T21:26:44.956183+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:44.956183+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5218304,"num_objects":1274,"num_object_clones":0,"num_object_copies":2548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17238,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17399","reported_seq":17507,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.097083+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:45.097083+0000","last_peered":"2024-09-15T21:26:45.097083+0000","last_clean":"2024-09-15T21:26:45.097083+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:45.097083+0000","last_undegraded":"2024-09-15T21:26:45.097083+0000","last_fullsized":"2024-09-15T21:26:45.097083+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5083136,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17591","reported_seq":24521,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.780011+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:45.780011+0000","last_peered":"2024-09-15T21:26:45.780011+0000","last_clean":"2024-09-15T21:26:45.780011+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:45.780011+0000","last_undegraded":"2024-09-15T21:26:45.780011+0000","last_fullsized":"2024-09-15T21:26:45.780011+0000","mapping_epoch":45,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5234712,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16314,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17349","reported_seq":17433,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.069167+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:45.069167+0000","last_peered":"2024-09-15T21:26:45.069167+0000","last_clean":"2024-09-15T21:26:45.069167+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:45.069167+0000","last_undegraded":"2024-09-15T21:26:45.069167+0000","last_fullsized":"2024-09-15T21:26:45.069167+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17349,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17532","reported_seq":17665,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.955891+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:44.955891+0000","last_peered":"2024-09-15T21:26:44.955891+0000","last_clean":"2024-09-15T21:26:44.955891+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:44.955891+0000","last_undegraded":"2024-09-15T21:26:44.955891+0000","last_fullsized":"2024-09-15T21:26:44.955891+0000","mapping_epoch":35,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878336,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17194","reported_seq":18022,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864447+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:44.864447+0000","last_peered":"2024-09-15T21:26:44.864447+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:44.864447+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14619,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17702","reported_seq":18453,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.838703+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:44.838703+0000","last_peered":"2024-09-15T21:26:44.838703+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:44.838703+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4923392,"num_objects":1202,"num_object_clones":0,"num_object_copies":2404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17186","reported_seq":17320,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.919640+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:44.919640+0000","last_peered":"2024-09-15T21:26:44.919640+0000","last_clean":"2024-09-15T21:26:44.919640+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:44.919640+0000","last_undegraded":"2024-09-15T21:26:44.919640+0000","last_fullsized":"2024-09-15T21:26:44.919640+0000","mapping_epoch":35,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4988928,"num_objects":1218,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14971,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17146","reported_seq":23974,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.716038+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:45.716038+0000","last_peered":"2024-09-15T21:26:45.716038+0000","last_clean":"2024-09-15T21:26:45.716038+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:45.716038+0000","last_undegraded":"2024-09-15T21:26:45.716038+0000","last_fullsized":"2024-09-15T21:26:45.716038+0000","mapping_epoch":49,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5033984,"num_objects":1229,"num_object_clones":0,"num_object_copies":2458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15732,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16876","reported_seq":17665,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864370+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:44.864370+0000","last_peered":"2024-09-15T21:26:44.864370+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:44.864370+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14402,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17029","reported_seq":22724,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.098930+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:45.098930+0000","last_peered":"2024-09-15T21:26:45.098930+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:45.098930+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5144576,"num_objects":1256,"num_object_clones":0,"num_object_copies":2512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17342","reported_seq":17756,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.106150+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:45.106150+0000","last_peered":"2024-09-15T21:26:45.106150+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:45.106150+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4939776,"num_objects":1206,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15741,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17488","reported_seq":18236,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.947044+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:44.947044+0000","last_peered":"2024-09-15T21:26:44.947044+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:44.947044+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5005312,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14795,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17225","reported_seq":24140,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.688988+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:45.688988+0000","last_peered":"2024-09-15T21:26:45.688988+0000","last_clean":"2024-09-15T21:26:45.688988+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:45.688988+0000","last_undegraded":"2024-09-15T21:26:45.688988+0000","last_fullsized":"2024-09-15T21:26:45.688988+0000","mapping_epoch":51,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14810,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17171","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.993938+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:44.993938+0000","last_peered":"2024-09-15T21:26:44.993938+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:44.993938+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17428","reported_seq":24526,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.752548+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:45.752548+0000","last_peered":"2024-09-15T21:26:45.752548+0000","last_clean":"2024-09-15T21:26:45.752548+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:45.752548+0000","last_undegraded":"2024-09-15T21:26:45.752548+0000","last_fullsized":"2024-09-15T21:26:45.752548+0000","mapping_epoch":47,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4874857,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15088,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":79737473,"num_objects":19469,"num_object_clones":0,"num_object_copies":38938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248310,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18394,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325091328,"data_stored":325034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433344,"kb_used_data":148972,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93938496,"statfs":{"total":96636764160,"available":96193019904,"internally_reserved":0,"allocated":152547328,"data_stored":152245926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532476,"kb_used_data":113516,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93839364,"statfs":{"total":96636764160,"available":96091508736,"internally_reserved":0,"allocated":116240384,"data_stored":115943054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483000,"kb_used_data":58408,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93888840,"statfs":{"total":96636764160,"available":96142172160,"internally_reserved":0,"allocated":59809792,"data_stored":59470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114249728,"data_stored":114246249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59052032,"data_stored":59006212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151789568,"data_stored":151782017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:47.668 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:48.441 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:48.485 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-15T21:26:46.529898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79737492,"num_objects":19471,"num_object_clones":0,"num_object_copies":38942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248312,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18395,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1448820,"kb_used_data":320896,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281666700,"statfs":{"total":289910292480,"available":288426700800,"internally_reserved":0,"allocated":328597504,"data_stored":327659101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1154927626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,4,9,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29040640,"num_objects":-7090,"num_object_clones":0,"num_object_copies":-14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-492,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002622"},"pg_stats":[{"pgid":"2.f","version":"52'17238","reported_seq":17998,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.956183+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:44.956183+0000","last_peered":"2024-09-15T21:26:44.956183+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:44.956183+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5218304,"num_objects":1274,"num_object_clones":0,"num_object_copies":2548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17238,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17399","reported_seq":17507,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.097083+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:45.097083+0000","last_peered":"2024-09-15T21:26:45.097083+0000","last_clean":"2024-09-15T21:26:45.097083+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:45.097083+0000","last_undegraded":"2024-09-15T21:26:45.097083+0000","last_fullsized":"2024-09-15T21:26:45.097083+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5083136,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17591","reported_seq":24521,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.780011+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:45.780011+0000","last_peered":"2024-09-15T21:26:45.780011+0000","last_clean":"2024-09-15T21:26:45.780011+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:45.780011+0000","last_undegraded":"2024-09-15T21:26:45.780011+0000","last_fullsized":"2024-09-15T21:26:45.780011+0000","mapping_epoch":45,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5234712,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16314,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17349","reported_seq":17433,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.069167+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:45.069167+0000","last_peered":"2024-09-15T21:26:45.069167+0000","last_clean":"2024-09-15T21:26:45.069167+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:45.069167+0000","last_undegraded":"2024-09-15T21:26:45.069167+0000","last_fullsized":"2024-09-15T21:26:45.069167+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17349,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17532","reported_seq":17665,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.955891+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:44.955891+0000","last_peered":"2024-09-15T21:26:44.955891+0000","last_clean":"2024-09-15T21:26:44.955891+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:44.955891+0000","last_undegraded":"2024-09-15T21:26:44.955891+0000","last_fullsized":"2024-09-15T21:26:44.955891+0000","mapping_epoch":35,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878336,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17194","reported_seq":18022,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864447+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:44.864447+0000","last_peered":"2024-09-15T21:26:44.864447+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:44.864447+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14619,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17702","reported_seq":18453,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.838703+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:44.838703+0000","last_peered":"2024-09-15T21:26:44.838703+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:44.838703+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4923392,"num_objects":1202,"num_object_clones":0,"num_object_copies":2404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17186","reported_seq":17320,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.919640+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:44.919640+0000","last_peered":"2024-09-15T21:26:44.919640+0000","last_clean":"2024-09-15T21:26:44.919640+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:44.919640+0000","last_undegraded":"2024-09-15T21:26:44.919640+0000","last_fullsized":"2024-09-15T21:26:44.919640+0000","mapping_epoch":35,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4988928,"num_objects":1218,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14971,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17146","reported_seq":23974,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.716038+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:45.716038+0000","last_peered":"2024-09-15T21:26:45.716038+0000","last_clean":"2024-09-15T21:26:45.716038+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:45.716038+0000","last_undegraded":"2024-09-15T21:26:45.716038+0000","last_fullsized":"2024-09-15T21:26:45.716038+0000","mapping_epoch":49,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5033984,"num_objects":1229,"num_object_clones":0,"num_object_copies":2458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15732,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16876","reported_seq":17665,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864370+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:44.864370+0000","last_peered":"2024-09-15T21:26:44.864370+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:44.864370+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14402,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17029","reported_seq":22724,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.098930+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:45.098930+0000","last_peered":"2024-09-15T21:26:45.098930+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:45.098930+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5144576,"num_objects":1256,"num_object_clones":0,"num_object_copies":2512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17342","reported_seq":17756,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.106150+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:45.106150+0000","last_peered":"2024-09-15T21:26:45.106150+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:45.106150+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4939776,"num_objects":1206,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15741,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17488","reported_seq":18236,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.947044+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:44.947044+0000","last_peered":"2024-09-15T21:26:44.947044+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:44.947044+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5005312,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14795,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17225","reported_seq":24140,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.688988+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:45.688988+0000","last_peered":"2024-09-15T21:26:45.688988+0000","last_clean":"2024-09-15T21:26:45.688988+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:45.688988+0000","last_undegraded":"2024-09-15T21:26:45.688988+0000","last_fullsized":"2024-09-15T21:26:45.688988+0000","mapping_epoch":51,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14810,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17171","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.993938+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:44.993938+0000","last_peered":"2024-09-15T21:26:44.993938+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:44.993938+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17428","reported_seq":24526,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.752548+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:45.752548+0000","last_peered":"2024-09-15T21:26:45.752548+0000","last_clean":"2024-09-15T21:26:45.752548+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:45.752548+0000","last_undegraded":"2024-09-15T21:26:45.752548+0000","last_fullsized":"2024-09-15T21:26:45.752548+0000","mapping_epoch":47,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4874857,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15088,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":79737473,"num_objects":19469,"num_object_clones":0,"num_object_copies":38938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248310,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18394,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325091328,"data_stored":325034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433344,"kb_used_data":148972,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93938496,"statfs":{"total":96636764160,"available":96193019904,"internally_reserved":0,"allocated":152547328,"data_stored":152245926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532476,"kb_used_data":113516,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93839364,"statfs":{"total":96636764160,"available":96091508736,"internally_reserved":0,"allocated":116240384,"data_stored":115943054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483000,"kb_used_data":58408,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93888840,"statfs":{"total":96636764160,"available":96142172160,"internally_reserved":0,"allocated":59809792,"data_stored":59470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114249728,"data_stored":114246249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59052032,"data_stored":59006212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151789568,"data_stored":151782017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'17238", 'reported_seq': 17998, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.956183+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:44.956183+0000', 'last_peered': '2024-09-15T21:26:44.956183+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:44.956183+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7200", 'ondisk_log_start': "46'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5218304, 'num_objects': 1274, 'num_object_clones': 0, 'num_object_copies': 2548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17238, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1757}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'17194", 'reported_seq': 18022, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.864447+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:44.864447+0000', 'last_peered': '2024-09-15T21:26:44.864447+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:44.864447+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7100", 'ondisk_log_start': "46'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4861952, 'num_objects': 1187, 'num_object_clones': 0, 'num_object_copies': 2374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14619, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'17702", 'reported_seq': 18453, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.838703+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:44.838703+0000', 'last_peered': '2024-09-15T21:26:44.838703+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:44.838703+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'7700", 'ondisk_log_start': "46'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4923392, 'num_objects': 1202, 'num_object_clones': 0, 'num_object_copies': 2404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15338, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1962}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:48.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'16876", 'reported_seq': 17665, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.864370+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:44.864370+0000', 'last_peered': '2024-09-15T21:26:44.864370+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:44.864370+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'6800", 'ondisk_log_start': "46'6800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4743168, 'num_objects': 1158, 'num_object_clones': 0, 'num_object_copies': 2316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14402, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1857}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:48.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'17029", 'reported_seq': 22724, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:45.098930+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:45.098930+0000', 'last_peered': '2024-09-15T21:26:45.098930+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:45.098930+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5144576, 'num_objects': 1256, 'num_object_clones': 0, 'num_object_copies': 2512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 483, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15580, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2785, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 773}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:48.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:48.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'17342", 'reported_seq': 17756, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:45.106150+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:45.106150+0000', 'last_peered': '2024-09-15T21:26:45.106150+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:45.106150+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'7300", 'ondisk_log_start': "46'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4939776, 'num_objects': 1206, 'num_object_clones': 0, 'num_object_copies': 2412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15741, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2000}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:48.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:48.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'17488", 'reported_seq': 18236, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.947044+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:44.947044+0000', 'last_peered': '2024-09-15T21:26:44.947044+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:44.947044+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'7400", 'ondisk_log_start': "46'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5005312, 'num_objects': 1222, 'num_object_clones': 0, 'num_object_copies': 2444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14795, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:48.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'17171", 'reported_seq': 17833, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.993938+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:44.993938+0000', 'last_peered': '2024-09-15T21:26:44.993938+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:44.993938+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7100", 'ondisk_log_start': "46'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4943872, 'num_objects': 1207, 'num_object_clones': 0, 'num_object_copies': 2414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14818, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1895}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:48.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:49.539 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:49.539 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22171/38942 objects degraded (56.933%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":622,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19471,"data_bytes":79737492,"bytes_used":1483591680,"bytes_avail":288426700800,"bytes_total":289910292480,"degraded_objects":22171,"degraded_total":38942,"degraded_ratio":0.56933388115659189,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":398},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:49.618 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:49.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:50.385 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:50.386 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:50.440 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-15T21:26:48.530336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":79737492,"num_objects":19471,"num_object_clones":0,"num_object_copies":38942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248312,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18395,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1448820,"kb_used_data":320896,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281666700,"statfs":{"total":289910292480,"available":288426700800,"internally_reserved":0,"allocated":328597504,"data_stored":327659101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1154927626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,4,9,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19570688,"num_objects":-4778,"num_object_clones":0,"num_object_copies":-9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4778,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"52'17238","reported_seq":17998,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.956183+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:44.956183+0000","last_peered":"2024-09-15T21:26:44.956183+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:44.956183+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5218304,"num_objects":1274,"num_object_clones":0,"num_object_copies":2548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17238,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17399","reported_seq":17507,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.097083+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:45.097083+0000","last_peered":"2024-09-15T21:26:45.097083+0000","last_clean":"2024-09-15T21:26:45.097083+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:45.097083+0000","last_undegraded":"2024-09-15T21:26:45.097083+0000","last_fullsized":"2024-09-15T21:26:45.097083+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5083136,"num_objects":1241,"num_object_clones":0,"num_object_copies":2482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17591","reported_seq":24521,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.780011+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:45.780011+0000","last_peered":"2024-09-15T21:26:45.780011+0000","last_clean":"2024-09-15T21:26:45.780011+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:45.780011+0000","last_undegraded":"2024-09-15T21:26:45.780011+0000","last_fullsized":"2024-09-15T21:26:45.780011+0000","mapping_epoch":45,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5234712,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16314,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17349","reported_seq":17433,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.069167+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:45.069167+0000","last_peered":"2024-09-15T21:26:45.069167+0000","last_clean":"2024-09-15T21:26:45.069167+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:45.069167+0000","last_undegraded":"2024-09-15T21:26:45.069167+0000","last_fullsized":"2024-09-15T21:26:45.069167+0000","mapping_epoch":36,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17349,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17532","reported_seq":17665,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.955891+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:44.955891+0000","last_peered":"2024-09-15T21:26:44.955891+0000","last_clean":"2024-09-15T21:26:44.955891+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:44.955891+0000","last_undegraded":"2024-09-15T21:26:44.955891+0000","last_fullsized":"2024-09-15T21:26:44.955891+0000","mapping_epoch":35,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878336,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17194","reported_seq":18022,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864447+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:44.864447+0000","last_peered":"2024-09-15T21:26:44.864447+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:44.864447+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14619,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17702","reported_seq":18453,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.838703+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:44.838703+0000","last_peered":"2024-09-15T21:26:44.838703+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:44.838703+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4923392,"num_objects":1202,"num_object_clones":0,"num_object_copies":2404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15338,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17186","reported_seq":17320,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.919640+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:44.919640+0000","last_peered":"2024-09-15T21:26:44.919640+0000","last_clean":"2024-09-15T21:26:44.919640+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:44.919640+0000","last_undegraded":"2024-09-15T21:26:44.919640+0000","last_fullsized":"2024-09-15T21:26:44.919640+0000","mapping_epoch":35,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4988928,"num_objects":1218,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14971,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17146","reported_seq":23974,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.716038+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:45.716038+0000","last_peered":"2024-09-15T21:26:45.716038+0000","last_clean":"2024-09-15T21:26:45.716038+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:45.716038+0000","last_undegraded":"2024-09-15T21:26:45.716038+0000","last_fullsized":"2024-09-15T21:26:45.716038+0000","mapping_epoch":49,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5033984,"num_objects":1229,"num_object_clones":0,"num_object_copies":2458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15732,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'16876","reported_seq":17665,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.864370+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:44.864370+0000","last_peered":"2024-09-15T21:26:44.864370+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:44.864370+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'6800","ondisk_log_start":"46'6800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4743168,"num_objects":1158,"num_object_clones":0,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14402,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1857}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17029","reported_seq":22724,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.098930+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:45.098930+0000","last_peered":"2024-09-15T21:26:45.098930+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:45.098930+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5144576,"num_objects":1256,"num_object_clones":0,"num_object_copies":2512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15580,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17342","reported_seq":17756,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:45.106150+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:45.106150+0000","last_peered":"2024-09-15T21:26:45.106150+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:45.106150+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4939776,"num_objects":1206,"num_object_clones":0,"num_object_copies":2412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15741,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17488","reported_seq":18236,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.947044+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:44.947044+0000","last_peered":"2024-09-15T21:26:44.947044+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:44.947044+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5005312,"num_objects":1222,"num_object_clones":0,"num_object_copies":2444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14795,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1908}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17225","reported_seq":24140,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.688988+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:45.688988+0000","last_peered":"2024-09-15T21:26:45.688988+0000","last_clean":"2024-09-15T21:26:45.688988+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:45.688988+0000","last_undegraded":"2024-09-15T21:26:45.688988+0000","last_fullsized":"2024-09-15T21:26:45.688988+0000","mapping_epoch":51,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4861952,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14810,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17171","reported_seq":17833,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.993938+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:44.993938+0000","last_peered":"2024-09-15T21:26:44.993938+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:44.993938+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17428","reported_seq":24526,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:45.752548+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:45.752548+0000","last_peered":"2024-09-15T21:26:45.752548+0000","last_clean":"2024-09-15T21:26:45.752548+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:45.752548+0000","last_undegraded":"2024-09-15T21:26:45.752548+0000","last_fullsized":"2024-09-15T21:26:45.752548+0000","mapping_epoch":47,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4874857,"num_objects":1191,"num_object_clones":0,"num_object_copies":2382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15088,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":79737473,"num_objects":19469,"num_object_clones":0,"num_object_copies":38938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248310,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18394,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325091328,"data_stored":325034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433344,"kb_used_data":148972,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93938496,"statfs":{"total":96636764160,"available":96193019904,"internally_reserved":0,"allocated":152547328,"data_stored":152245926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532476,"kb_used_data":113516,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93839364,"statfs":{"total":96636764160,"available":96091508736,"internally_reserved":0,"allocated":116240384,"data_stored":115943054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,7,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483000,"kb_used_data":58408,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93888840,"statfs":{"total":96636764160,"available":96142172160,"internally_reserved":0,"allocated":59809792,"data_stored":59470121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114249728,"data_stored":114246249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59052032,"data_stored":59006212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151789568,"data_stored":151782017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:53.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:54.270 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:54.271 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:54.313 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-15T21:26:52.531195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69763732,"num_objects":17036,"num_object_clones":0,"num_object_copies":34072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250747,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18550,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429316,"kb_used_data":301384,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281686204,"statfs":{"total":289910292480,"available":288446672896,"internally_reserved":0,"allocated":308617216,"data_stored":307678813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1154927626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,6,9,9,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22315008,"num_objects":-5448,"num_object_clones":0,"num_object_copies":-10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5448,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"2.f","version":"52'17378","reported_seq":18139,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.981063+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:49.981063+0000","last_peered":"2024-09-15T21:26:49.981063+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:49.981063+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":1134,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17378,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17552","reported_seq":17663,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.145580+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:50.145580+0000","last_peered":"2024-09-15T21:26:50.145580+0000","last_clean":"2024-09-15T21:26:50.145580+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:50.145580+0000","last_undegraded":"2024-09-15T21:26:50.145580+0000","last_fullsized":"2024-09-15T21:26:50.145580+0000","mapping_epoch":36,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17733","reported_seq":24663,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.758264+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:50.758264+0000","last_peered":"2024-09-15T21:26:50.758264+0000","last_clean":"2024-09-15T21:26:50.758264+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:50.758264+0000","last_undegraded":"2024-09-15T21:26:50.758264+0000","last_fullsized":"2024-09-15T21:26:50.758264+0000","mapping_epoch":45,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653080,"num_objects":1137,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16456,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17527","reported_seq":17611,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.130620+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:50.130620+0000","last_peered":"2024-09-15T21:26:50.130620+0000","last_clean":"2024-09-15T21:26:50.130620+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:50.130620+0000","last_undegraded":"2024-09-15T21:26:50.130620+0000","last_fullsized":"2024-09-15T21:26:50.130620+0000","mapping_epoch":36,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4272128,"num_objects":1043,"num_object_clones":0,"num_object_copies":2086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17682","reported_seq":17815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.989045+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:49.989045+0000","last_peered":"2024-09-15T21:26:49.989045+0000","last_clean":"2024-09-15T21:26:49.989045+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:49.989045+0000","last_undegraded":"2024-09-15T21:26:49.989045+0000","last_fullsized":"2024-09-15T21:26:49.989045+0000","mapping_epoch":35,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17344","reported_seq":18172,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.956436+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:49.956436+0000","last_peered":"2024-09-15T21:26:49.956436+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:49.956436+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4247552,"num_objects":1037,"num_object_clones":0,"num_object_copies":2074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14769,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17839","reported_seq":18590,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.009283+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:50.009283+0000","last_peered":"2024-09-15T21:26:50.009283+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:50.009283+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362240,"num_objects":1065,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15475,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17336","reported_seq":17470,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.972287+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:49.972287+0000","last_peered":"2024-09-15T21:26:49.972287+0000","last_clean":"2024-09-15T21:26:49.972287+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:49.972287+0000","last_undegraded":"2024-09-15T21:26:49.972287+0000","last_fullsized":"2024-09-15T21:26:49.972287+0000","mapping_epoch":35,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4374528,"num_objects":1068,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15121,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17308","reported_seq":24136,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.777089+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:50.777089+0000","last_peered":"2024-09-15T21:26:50.777089+0000","last_clean":"2024-09-15T21:26:50.777089+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:50.777089+0000","last_undegraded":"2024-09-15T21:26:50.777089+0000","last_fullsized":"2024-09-15T21:26:50.777089+0000","mapping_epoch":49,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370432,"num_objects":1067,"num_object_clones":0,"num_object_copies":2134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15894,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'17036","reported_seq":17824,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.821992+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:49.821992+0000","last_peered":"2024-09-15T21:26:49.821992+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:49.821992+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4087808,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2017}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17172","reported_seq":23177,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.134189+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:50.134189+0000","last_peered":"2024-09-15T21:26:50.134189+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:50.134189+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4558848,"num_objects":1113,"num_object_clones":0,"num_object_copies":2226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15723,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2940,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17494","reported_seq":17911,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.083906+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:50.083906+0000","last_peered":"2024-09-15T21:26:50.083906+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:50.083906+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4317184,"num_objects":1054,"num_object_clones":0,"num_object_copies":2108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15893,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2152}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17615","reported_seq":18362,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.884582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:49.884582+0000","last_peered":"2024-09-15T21:26:49.884582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:49.884582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4485120,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14922,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17405","reported_seq":24322,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.790784+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:50.790784+0000","last_peered":"2024-09-15T21:26:50.790784+0000","last_clean":"2024-09-15T21:26:50.790784+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:50.790784+0000","last_undegraded":"2024-09-15T21:26:50.790784+0000","last_fullsized":"2024-09-15T21:26:50.790784+0000","mapping_epoch":51,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4124672,"num_objects":1007,"num_object_clones":0,"num_object_copies":2014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14990,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17312","reported_seq":17976,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.985152+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:49.985152+0000","last_peered":"2024-09-15T21:26:49.985152+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:49.985152+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14959,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17598","reported_seq":24697,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.796830+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:50.796830+0000","last_peered":"2024-09-15T21:26:50.796830+0000","last_clean":"2024-09-15T21:26:50.796830+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:50.796830+0000","last_undegraded":"2024-09-15T21:26:50.796830+0000","last_fullsized":"2024-09-15T21:26:50.796830+0000","mapping_epoch":47,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4178537,"num_objects":1021,"num_object_clones":0,"num_object_copies":2042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15258,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":69763713,"num_objects":17034,"num_object_clones":0,"num_object_copies":34068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250745,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18549,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305176576,"data_stored":305119726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953588,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432580,"kb_used_data":144048,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93939260,"statfs":{"total":96636764160,"available":96193802240,"internally_reserved":0,"allocated":147505152,"data_stored":147203750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149815,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512608,"kb_used_data":106124,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93859232,"statfs":{"total":96636764160,"available":96111853568,"internally_reserved":0,"allocated":108670976,"data_stored":108373646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,1,3,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626534,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484128,"kb_used_data":51212,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93887712,"statfs":{"total":96636764160,"available":96141017088,"internally_reserved":0,"allocated":52441088,"data_stored":52101417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":443281068},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106786816,"data_stored":106783337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51646464,"data_stored":51600644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146743296,"data_stored":146735745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:54.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:55.130 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:55.132 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:55.169 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-15T21:26:54.531628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":69763732,"num_objects":17036,"num_object_clones":0,"num_object_copies":34072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250747,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18550,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429316,"kb_used_data":301384,"kb_used_omap":12,"kb_used_meta":1127859,"kb_avail":281686204,"statfs":{"total":289910292480,"available":288446672896,"internally_reserved":0,"allocated":308617216,"data_stored":307678813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1154927626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,6,9,9,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19681280,"num_objects":-4805,"num_object_clones":0,"num_object_copies":-9610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002595"},"pg_stats":[{"pgid":"2.f","version":"52'17378","reported_seq":18139,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.981063+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:49.981063+0000","last_peered":"2024-09-15T21:26:49.981063+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:49.981063+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4644864,"num_objects":1134,"num_object_clones":0,"num_object_copies":2268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17378,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17552","reported_seq":17663,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.145580+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:50.145580+0000","last_peered":"2024-09-15T21:26:50.145580+0000","last_clean":"2024-09-15T21:26:50.145580+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:50.145580+0000","last_undegraded":"2024-09-15T21:26:50.145580+0000","last_fullsized":"2024-09-15T21:26:50.145580+0000","mapping_epoch":36,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17733","reported_seq":24663,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.758264+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:50.758264+0000","last_peered":"2024-09-15T21:26:50.758264+0000","last_clean":"2024-09-15T21:26:50.758264+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:50.758264+0000","last_undegraded":"2024-09-15T21:26:50.758264+0000","last_fullsized":"2024-09-15T21:26:50.758264+0000","mapping_epoch":45,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653080,"num_objects":1137,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16456,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17527","reported_seq":17611,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.130620+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:50.130620+0000","last_peered":"2024-09-15T21:26:50.130620+0000","last_clean":"2024-09-15T21:26:50.130620+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:50.130620+0000","last_undegraded":"2024-09-15T21:26:50.130620+0000","last_fullsized":"2024-09-15T21:26:50.130620+0000","mapping_epoch":36,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4272128,"num_objects":1043,"num_object_clones":0,"num_object_copies":2086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17682","reported_seq":17815,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.989045+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:49.989045+0000","last_peered":"2024-09-15T21:26:49.989045+0000","last_clean":"2024-09-15T21:26:49.989045+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:49.989045+0000","last_undegraded":"2024-09-15T21:26:49.989045+0000","last_fullsized":"2024-09-15T21:26:49.989045+0000","mapping_epoch":35,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17344","reported_seq":18172,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.956436+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:49.956436+0000","last_peered":"2024-09-15T21:26:49.956436+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:49.956436+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4247552,"num_objects":1037,"num_object_clones":0,"num_object_copies":2074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14769,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17839","reported_seq":18590,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.009283+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:50.009283+0000","last_peered":"2024-09-15T21:26:50.009283+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:50.009283+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362240,"num_objects":1065,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15475,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17336","reported_seq":17470,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.972287+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:49.972287+0000","last_peered":"2024-09-15T21:26:49.972287+0000","last_clean":"2024-09-15T21:26:49.972287+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:49.972287+0000","last_undegraded":"2024-09-15T21:26:49.972287+0000","last_fullsized":"2024-09-15T21:26:49.972287+0000","mapping_epoch":35,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4374528,"num_objects":1068,"num_object_clones":0,"num_object_copies":2136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15121,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17308","reported_seq":24136,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.777089+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:50.777089+0000","last_peered":"2024-09-15T21:26:50.777089+0000","last_clean":"2024-09-15T21:26:50.777089+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:50.777089+0000","last_undegraded":"2024-09-15T21:26:50.777089+0000","last_fullsized":"2024-09-15T21:26:50.777089+0000","mapping_epoch":49,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370432,"num_objects":1067,"num_object_clones":0,"num_object_copies":2134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15894,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'17036","reported_seq":17824,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.821992+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:49.821992+0000","last_peered":"2024-09-15T21:26:49.821992+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:49.821992+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7000","ondisk_log_start":"46'7000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4087808,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14562,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2017}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17172","reported_seq":23177,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.134189+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:50.134189+0000","last_peered":"2024-09-15T21:26:50.134189+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:50.134189+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4558848,"num_objects":1113,"num_object_clones":0,"num_object_copies":2226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15723,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2940,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17494","reported_seq":17911,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:50.083906+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:50.083906+0000","last_peered":"2024-09-15T21:26:50.083906+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:50.083906+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4317184,"num_objects":1054,"num_object_clones":0,"num_object_copies":2108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15893,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2152}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17615","reported_seq":18362,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.884582+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:49.884582+0000","last_peered":"2024-09-15T21:26:49.884582+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:49.884582+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4485120,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14922,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17405","reported_seq":24322,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.790784+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:50.790784+0000","last_peered":"2024-09-15T21:26:50.790784+0000","last_clean":"2024-09-15T21:26:50.790784+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:50.790784+0000","last_undegraded":"2024-09-15T21:26:50.790784+0000","last_fullsized":"2024-09-15T21:26:50.790784+0000","mapping_epoch":51,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4124672,"num_objects":1007,"num_object_clones":0,"num_object_copies":2014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14990,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17312","reported_seq":17976,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.985152+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:49.985152+0000","last_peered":"2024-09-15T21:26:49.985152+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:49.985152+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7300","ondisk_log_start":"46'7300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14959,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2036}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17598","reported_seq":24697,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:50.796830+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:50.796830+0000","last_peered":"2024-09-15T21:26:50.796830+0000","last_clean":"2024-09-15T21:26:50.796830+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:50.796830+0000","last_undegraded":"2024-09-15T21:26:50.796830+0000","last_fullsized":"2024-09-15T21:26:50.796830+0000","mapping_epoch":47,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4178537,"num_objects":1021,"num_object_clones":0,"num_object_copies":2042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15258,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":69763713,"num_objects":17034,"num_object_clones":0,"num_object_copies":34068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250745,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18549,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305176576,"data_stored":305119726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953588,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432580,"kb_used_data":144048,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93939260,"statfs":{"total":96636764160,"available":96193802240,"internally_reserved":0,"allocated":147505152,"data_stored":147203750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149815,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512608,"kb_used_data":106124,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93859232,"statfs":{"total":96636764160,"available":96111853568,"internally_reserved":0,"allocated":108670976,"data_stored":108373646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":416214705},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,1,3,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626534,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484128,"kb_used_data":51212,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93887712,"statfs":{"total":96636764160,"available":96141017088,"internally_reserved":0,"allocated":52441088,"data_stored":52101417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":443281068},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106786816,"data_stored":106783337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51646464,"data_stored":51600644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146743296,"data_stored":146735745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "52'17378", 'reported_seq': 18139, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.981063+0000', 'last_change': '2024-09-15T21:18:43.853730+0000', 'last_active': '2024-09-15T21:26:49.981063+0000', 'last_peered': '2024-09-15T21:26:49.981063+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:26:49.981063+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7300", 'ondisk_log_start': "46'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4644864, 'num_objects': 1134, 'num_object_clones': 0, 'num_object_copies': 2268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17378, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1897}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "52'17344", 'reported_seq': 18172, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.956436+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:26:49.956436+0000', 'last_peered': '2024-09-15T21:26:49.956436+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:26:49.956436+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7300", 'ondisk_log_start': "46'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4247552, 'num_objects': 1037, 'num_object_clones': 0, 'num_object_copies': 2074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14769, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1992}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "52'17839", 'reported_seq': 18590, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:50.009283+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:50.009283+0000', 'last_peered': '2024-09-15T21:26:50.009283+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:50.009283+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4362240, 'num_objects': 1065, 'num_object_clones': 0, 'num_object_copies': 2130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15475, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "52'17036", 'reported_seq': 17824, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.821992+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:26:49.821992+0000', 'last_peered': '2024-09-15T21:26:49.821992+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:26:49.821992+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7000", 'ondisk_log_start': "46'7000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4087808, 'num_objects': 998, 'num_object_clones': 0, 'num_object_copies': 1996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14562, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2017}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:55.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "52'17172", 'reported_seq': 23177, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:50.134189+0000', 'last_change': '2024-09-15T21:25:02.567147+0000', 'last_active': '2024-09-15T21:26:50.134189+0000', 'last_peered': '2024-09-15T21:26:50.134189+0000', 'last_clean': '2024-09-15T21:18:41.345310+0000', 'last_became_active': '2024-09-15T21:18:43.836055+0000', 'last_became_peered': '2024-09-15T21:18:43.836055+0000', 'last_unstale': '2024-09-15T21:26:50.134189+0000', 'last_undegraded': '2024-09-15T21:18:43.442812+0000', 'last_fullsized': '2024-09-15T21:18:43.369147+0000', 'mapping_epoch': 43, 'log_start': "46'7100", 'ondisk_log_start': "46'7100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1469", 'last_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:09.166204+0000', 'objects_scrubbed': 1444, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 427, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4558848, 'num_objects': 1113, 'num_object_clones': 0, 'num_object_copies': 2226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 328, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15723, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2940, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 785}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "52'17494", 'reported_seq': 17911, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:50.083906+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:26:50.083906+0000', 'last_peered': '2024-09-15T21:26:50.083906+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:26:50.083906+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'7400", 'ondisk_log_start': "46'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4317184, 'num_objects': 1054, 'num_object_clones': 0, 'num_object_copies': 2108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15893, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2152}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "52'17615", 'reported_seq': 18362, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.884582+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:26:49.884582+0000', 'last_peered': '2024-09-15T21:26:49.884582+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:26:49.884582+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'7600", 'ondisk_log_start': "46'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4485120, 'num_objects': 1095, 'num_object_clones': 0, 'num_object_copies': 2190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1095, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14922, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:55.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "52'17312", 'reported_seq': 17976, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.985152+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:49.985152+0000', 'last_peered': '2024-09-15T21:26:49.985152+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:49.985152+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7300", 'ondisk_log_start': "46'7300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4366336, 'num_objects': 1066, 'num_object_clones': 0, 'num_object_copies': 2132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14959, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:55.172 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:55.825 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:55.825 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22016/34072 objects degraded (64.616%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":628,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":17036,"data_bytes":69763732,"bytes_used":1463619584,"bytes_avail":288446672896,"bytes_total":289910292480,"degraded_objects":22016,"degraded_total":34072,"degraded_ratio":0.64616107067386708,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:55.894 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:55.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:56.727 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:26:56.730 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:26:56.787 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-15T21:26:56.532072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":61080212,"num_objects":14916,"num_object_clones":0,"num_object_copies":29832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":252867,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18772,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429140,"kb_used_data":284572,"kb_used_omap":12,"kb_used_meta":1144499,"kb_avail":281686380,"statfs":{"total":289910292480,"available":288446853120,"internally_reserved":0,"allocated":291401728,"data_stored":290463325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1171966986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,3,7,10,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28364800,"num_objects":-6925,"num_object_clones":0,"num_object_copies":-13850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6925,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"2.f","version":"52'17527","reported_seq":18288,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.988454+0000","last_change":"2024-09-15T21:18:43.853730+0000","last_active":"2024-09-15T21:26:54.988454+0000","last_peered":"2024-09-15T21:26:54.988454+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:26:54.988454+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17527,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"52'17686","reported_seq":17796,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.111788+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:26:55.111788+0000","last_peered":"2024-09-15T21:26:55.111788+0000","last_clean":"2024-09-15T21:26:55.111788+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:26:55.111788+0000","last_undegraded":"2024-09-15T21:26:55.111788+0000","last_fullsized":"2024-09-15T21:26:55.111788+0000","mapping_epoch":36,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3907584,"num_objects":954,"num_object_clones":0,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16412,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17880","reported_seq":24810,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.884782+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:55.884782+0000","last_peered":"2024-09-15T21:26:55.884782+0000","last_clean":"2024-09-15T21:26:55.884782+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:55.884782+0000","last_undegraded":"2024-09-15T21:26:55.884782+0000","last_fullsized":"2024-09-15T21:26:55.884782+0000","mapping_epoch":45,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4050968,"num_objects":990,"num_object_clones":0,"num_object_copies":1980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16603,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"52'17666","reported_seq":17750,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.106341+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:26:55.106341+0000","last_peered":"2024-09-15T21:26:55.106341+0000","last_clean":"2024-09-15T21:26:55.106341+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:26:55.106341+0000","last_undegraded":"2024-09-15T21:26:55.106341+0000","last_fullsized":"2024-09-15T21:26:55.106341+0000","mapping_epoch":36,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3702784,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17666,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"52'17808","reported_seq":17941,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:54.960366+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:26:54.960366+0000","last_peered":"2024-09-15T21:26:54.960366+0000","last_clean":"2024-09-15T21:26:54.960366+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:26:54.960366+0000","last_undegraded":"2024-09-15T21:26:54.960366+0000","last_fullsized":"2024-09-15T21:26:54.960366+0000","mapping_epoch":35,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15666,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"52'17473","reported_seq":18301,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:55.031041+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:26:55.031041+0000","last_peered":"2024-09-15T21:26:55.031041+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:26:55.031041+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3719168,"num_objects":908,"num_object_clones":0,"num_object_copies":1816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14898,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"52'17963","reported_seq":18713,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.976289+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:54.976289+0000","last_peered":"2024-09-15T21:26:54.976289+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:54.976289+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3854336,"num_objects":941,"num_object_clones":0,"num_object_copies":1882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15599,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2223}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"52'17485","reported_seq":17619,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.015713+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:26:55.015713+0000","last_peered":"2024-09-15T21:26:55.015713+0000","last_clean":"2024-09-15T21:26:55.015713+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:26:55.015713+0000","last_undegraded":"2024-09-15T21:26:55.015713+0000","last_fullsized":"2024-09-15T21:26:55.015713+0000","mapping_epoch":35,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3764224,"num_objects":919,"num_object_clones":0,"num_object_copies":1838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15270,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17423","reported_seq":24251,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.788457+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:55.788457+0000","last_peered":"2024-09-15T21:26:55.788457+0000","last_clean":"2024-09-15T21:26:55.788457+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:55.788457+0000","last_undegraded":"2024-09-15T21:26:55.788457+0000","last_fullsized":"2024-09-15T21:26:55.788457+0000","mapping_epoch":49,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899392,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.727067+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:25:03.727067+0000","last_peered":"2024-09-15T21:25:03.727067+0000","last_clean":"2024-09-15T21:25:03.727067+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:25:03.727067+0000","last_undegraded":"2024-09-15T21:25:03.727067+0000","last_fullsized":"2024-09-15T21:25:03.727067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"52'17166","reported_seq":17955,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.984859+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:26:54.984859+0000","last_peered":"2024-09-15T21:26:54.984859+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:26:54.984859+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7100","ondisk_log_start":"46'7100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3555328,"num_objects":868,"num_object_clones":0,"num_object_copies":1736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14692,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2147}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":115,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.725100+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:25:03.725100+0000","last_peered":"2024-09-15T21:25:03.725100+0000","last_clean":"2024-09-15T21:25:03.725100+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:25:03.725100+0000","last_undegraded":"2024-09-15T21:25:03.725100+0000","last_fullsized":"2024-09-15T21:25:03.725100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"52'17296","reported_seq":23745,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:55.150170+0000","last_change":"2024-09-15T21:25:02.567147+0000","last_active":"2024-09-15T21:26:55.150170+0000","last_peered":"2024-09-15T21:26:55.150170+0000","last_clean":"2024-09-15T21:18:41.345310+0000","last_became_active":"2024-09-15T21:18:43.836055+0000","last_became_peered":"2024-09-15T21:18:43.836055+0000","last_unstale":"2024-09-15T21:26:55.150170+0000","last_undegraded":"2024-09-15T21:18:43.442812+0000","last_fullsized":"2024-09-15T21:18:43.369147+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4050944,"num_objects":989,"num_object_clones":0,"num_object_copies":1978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":883}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"52'17615","reported_seq":18032,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:55.112231+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:26:55.112231+0000","last_peered":"2024-09-15T21:26:55.112231+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:26:55.112231+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3821568,"num_objects":933,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16014,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"52'17763","reported_seq":18509,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:55.046014+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:26:55.046014+0000","last_peered":"2024-09-15T21:26:55.046014+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:26:55.046014+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3878912,"num_objects":947,"num_object_clones":0,"num_object_copies":1894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15070,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17543","reported_seq":24461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.767088+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:55.767088+0000","last_peered":"2024-09-15T21:26:55.767088+0000","last_clean":"2024-09-15T21:26:55.767088+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:55.767088+0000","last_undegraded":"2024-09-15T21:26:55.767088+0000","last_fullsized":"2024-09-15T21:26:55.767088+0000","mapping_epoch":51,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15128,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730318+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:25:03.730318+0000","last_peered":"2024-09-15T21:25:03.730318+0000","last_clean":"2024-09-15T21:25:03.730318+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:25:03.730318+0000","last_undegraded":"2024-09-15T21:25:03.730318+0000","last_fullsized":"2024-09-15T21:25:03.730318+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"52'17448","reported_seq":18112,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.994447+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:54.994447+0000","last_peered":"2024-09-15T21:26:54.994447+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:54.994447+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3809280,"num_objects":930,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15095,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2172}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17709","reported_seq":24808,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.814183+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:55.814183+0000","last_peered":"2024-09-15T21:26:55.814183+0000","last_clean":"2024-09-15T21:26:55.814183+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:55.814183+0000","last_undegraded":"2024-09-15T21:26:55.814183+0000","last_fullsized":"2024-09-15T21:26:55.814183+0000","mapping_epoch":47,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723881,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":61080193,"num_objects":14914,"num_object_clones":0,"num_object_copies":29828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":252865,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18771,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":287879168,"data_stored":287822318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432640,"kb_used_data":139952,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93939200,"statfs":{"total":96636764160,"available":96193740800,"internally_reserved":0,"allocated":143310848,"data_stored":143009446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149816,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514240,"kb_used_data":99436,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93857600,"statfs":{"total":96636764160,"available":96110182400,"internally_reserved":0,"allocated":101822464,"data_stored":101525134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482260,"kb_used_data":45184,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93889580,"statfs":{"total":96636764160,"available":96142929920,"internally_reserved":0,"allocated":46268416,"data_stored":45928745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100048896,"data_stored":100045417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45355008,"data_stored":45309188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142475264,"data_stored":142467713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:59.790 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:00.816 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:00.817 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:00.887 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-15T21:27:00.532920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54456980,"num_objects":13299,"num_object_clones":0,"num_object_copies":26598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254484,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18970,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1426868,"kb_used_data":273964,"kb_used_omap":12,"kb_used_meta":1152819,"kb_avail":281688652,"statfs":{"total":289910292480,"available":288449179648,"internally_reserved":0,"allocated":280539136,"data_stored":279579543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1180486666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,8,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15306752,"num_objects":-3737,"num_object_clones":0,"num_object_copies":-7474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001725"},"pg_stats":[{"pgid":"2.f","version":"54'17671","reported_seq":18623,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.047948+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:00.047948+0000","last_peered":"2024-09-15T21:27:00.047948+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:00.047948+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3444736,"num_objects":841,"num_object_clones":0,"num_object_copies":1682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17671,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2098}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17814","reported_seq":17929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.132008+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:00.132008+0000","last_peered":"2024-09-15T21:27:00.132008+0000","last_clean":"2024-09-15T21:27:00.132008+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:00.132008+0000","last_undegraded":"2024-09-15T21:27:00.132008+0000","last_fullsized":"2024-09-15T21:27:00.132008+0000","mapping_epoch":36,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17880","reported_seq":24810,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.884782+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:55.884782+0000","last_peered":"2024-09-15T21:26:55.884782+0000","last_clean":"2024-09-15T21:26:55.884782+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:55.884782+0000","last_undegraded":"2024-09-15T21:26:55.884782+0000","last_fullsized":"2024-09-15T21:26:55.884782+0000","mapping_epoch":45,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4050968,"num_objects":990,"num_object_clones":0,"num_object_copies":1980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16603,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17799","reported_seq":17887,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.129025+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:00.129025+0000","last_peered":"2024-09-15T21:27:00.129025+0000","last_clean":"2024-09-15T21:27:00.129025+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:00.129025+0000","last_undegraded":"2024-09-15T21:27:00.129025+0000","last_fullsized":"2024-09-15T21:27:00.129025+0000","mapping_epoch":36,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3158016,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17799,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'17944","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.000745+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:00.000745+0000","last_peered":"2024-09-15T21:27:00.000745+0000","last_clean":"2024-09-15T21:27:00.000745+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:00.000745+0000","last_undegraded":"2024-09-15T21:27:00.000745+0000","last_fullsized":"2024-09-15T21:27:00.000745+0000","mapping_epoch":35,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3190784,"num_objects":779,"num_object_clones":0,"num_object_copies":1558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15802,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17604","reported_seq":18434,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.056637+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:00.056637+0000","last_peered":"2024-09-15T21:27:00.056637+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:00.056637+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3182592,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18109","reported_seq":18859,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.987880+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:59.987880+0000","last_peered":"2024-09-15T21:26:59.987880+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:59.987880+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3256320,"num_objects":795,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17623","reported_seq":17761,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.004015+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:00.004015+0000","last_peered":"2024-09-15T21:27:00.004015+0000","last_clean":"2024-09-15T21:27:00.004015+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:00.004015+0000","last_undegraded":"2024-09-15T21:27:00.004015+0000","last_fullsized":"2024-09-15T21:27:00.004015+0000","mapping_epoch":35,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15408,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17423","reported_seq":24251,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.788457+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:55.788457+0000","last_peered":"2024-09-15T21:26:55.788457+0000","last_clean":"2024-09-15T21:26:55.788457+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:55.788457+0000","last_undegraded":"2024-09-15T21:26:55.788457+0000","last_fullsized":"2024-09-15T21:26:55.788457+0000","mapping_epoch":49,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899392,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17282","reported_seq":18074,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.068491+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:00.068491+0000","last_peered":"2024-09-15T21:27:00.068491+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:00.068491+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":752,"num_object_clones":0,"num_object_copies":1504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17439","reported_seq":24106,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.165028+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:00.165028+0000","last_peered":"2024-09-15T21:27:00.165028+0000","last_clean":"2024-09-15T21:27:00.165028+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:00.165028+0000","last_undegraded":"2024-09-15T21:27:00.165028+0000","last_fullsized":"2024-09-15T21:27:00.165028+0000","mapping_epoch":53,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3465216,"num_objects":846,"num_object_clones":0,"num_object_copies":1692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17757","reported_seq":18178,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.135990+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:00.135990+0000","last_peered":"2024-09-15T21:27:00.135990+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:00.135990+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3239936,"num_objects":791,"num_object_clones":0,"num_object_copies":1582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16156,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'17891","reported_seq":18643,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.032733+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:00.032733+0000","last_peered":"2024-09-15T21:27:00.032733+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:00.032733+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17543","reported_seq":24461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.767088+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:55.767088+0000","last_peered":"2024-09-15T21:26:55.767088+0000","last_clean":"2024-09-15T21:26:55.767088+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:55.767088+0000","last_undegraded":"2024-09-15T21:26:55.767088+0000","last_fullsized":"2024-09-15T21:26:55.767088+0000","mapping_epoch":51,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15128,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17580","reported_seq":18248,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.895266+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:59.895266+0000","last_peered":"2024-09-15T21:26:59.895266+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:59.895266+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15227,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17709","reported_seq":24808,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.814183+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:55.814183+0000","last_peered":"2024-09-15T21:26:55.814183+0000","last_clean":"2024-09-15T21:26:55.814183+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:55.814183+0000","last_undegraded":"2024-09-15T21:26:55.814183+0000","last_fullsized":"2024-09-15T21:26:55.814183+0000","mapping_epoch":47,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723881,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":54456961,"num_objects":13297,"num_object_clones":0,"num_object_copies":26594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254482,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18969,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":276787200,"data_stored":276730350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432396,"kb_used_data":135536,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93939444,"statfs":{"total":96636764160,"available":96193990656,"internally_reserved":0,"allocated":138788864,"data_stored":138476867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512212,"kb_used_data":93244,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93859628,"statfs":{"total":96636764160,"available":96112259072,"internally_reserved":0,"allocated":95481856,"data_stored":95173931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482260,"kb_used_data":45184,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93889580,"statfs":{"total":96636764160,"available":96142929920,"internally_reserved":0,"allocated":46268416,"data_stored":45928745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93564928,"data_stored":93561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45355008,"data_stored":45309188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137867264,"data_stored":137859713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:00.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:01.760 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:01.762 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:01.799 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-15T21:27:00.532920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54456980,"num_objects":13299,"num_object_clones":0,"num_object_copies":26598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254484,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18970,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1426868,"kb_used_data":273964,"kb_used_omap":12,"kb_used_meta":1152819,"kb_avail":281688652,"statfs":{"total":289910292480,"available":288449179648,"internally_reserved":0,"allocated":280539136,"data_stored":279579543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1180486666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,8,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15306752,"num_objects":-3737,"num_object_clones":0,"num_object_copies":-7474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001725"},"pg_stats":[{"pgid":"2.f","version":"54'17671","reported_seq":18623,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.047948+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:00.047948+0000","last_peered":"2024-09-15T21:27:00.047948+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:00.047948+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3444736,"num_objects":841,"num_object_clones":0,"num_object_copies":1682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17671,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2098}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17814","reported_seq":17929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.132008+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:00.132008+0000","last_peered":"2024-09-15T21:27:00.132008+0000","last_clean":"2024-09-15T21:27:00.132008+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:00.132008+0000","last_undegraded":"2024-09-15T21:27:00.132008+0000","last_fullsized":"2024-09-15T21:27:00.132008+0000","mapping_epoch":36,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"52'17880","reported_seq":24810,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.884782+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:26:55.884782+0000","last_peered":"2024-09-15T21:26:55.884782+0000","last_clean":"2024-09-15T21:26:55.884782+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:26:55.884782+0000","last_undegraded":"2024-09-15T21:26:55.884782+0000","last_fullsized":"2024-09-15T21:26:55.884782+0000","mapping_epoch":45,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4050968,"num_objects":990,"num_object_clones":0,"num_object_copies":1980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16603,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17799","reported_seq":17887,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.129025+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:00.129025+0000","last_peered":"2024-09-15T21:27:00.129025+0000","last_clean":"2024-09-15T21:27:00.129025+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:00.129025+0000","last_undegraded":"2024-09-15T21:27:00.129025+0000","last_fullsized":"2024-09-15T21:27:00.129025+0000","mapping_epoch":36,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3158016,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17799,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'17944","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.000745+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:00.000745+0000","last_peered":"2024-09-15T21:27:00.000745+0000","last_clean":"2024-09-15T21:27:00.000745+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:00.000745+0000","last_undegraded":"2024-09-15T21:27:00.000745+0000","last_fullsized":"2024-09-15T21:27:00.000745+0000","mapping_epoch":35,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3190784,"num_objects":779,"num_object_clones":0,"num_object_copies":1558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15802,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17604","reported_seq":18434,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.056637+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:00.056637+0000","last_peered":"2024-09-15T21:27:00.056637+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:00.056637+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3182592,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18109","reported_seq":18859,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.987880+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:59.987880+0000","last_peered":"2024-09-15T21:26:59.987880+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:59.987880+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3256320,"num_objects":795,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17623","reported_seq":17761,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.004015+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:00.004015+0000","last_peered":"2024-09-15T21:27:00.004015+0000","last_clean":"2024-09-15T21:27:00.004015+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:00.004015+0000","last_undegraded":"2024-09-15T21:27:00.004015+0000","last_fullsized":"2024-09-15T21:27:00.004015+0000","mapping_epoch":35,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15408,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.724915+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:25:03.724915+0000","last_peered":"2024-09-15T21:25:03.724915+0000","last_clean":"2024-09-15T21:25:03.724915+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:25:03.724915+0000","last_undegraded":"2024-09-15T21:25:03.724915+0000","last_fullsized":"2024-09-15T21:25:03.724915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"52'17423","reported_seq":24251,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.788457+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:26:55.788457+0000","last_peered":"2024-09-15T21:26:55.788457+0000","last_clean":"2024-09-15T21:26:55.788457+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:26:55.788457+0000","last_undegraded":"2024-09-15T21:26:55.788457+0000","last_fullsized":"2024-09-15T21:26:55.788457+0000","mapping_epoch":49,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899392,"num_objects":952,"num_object_clones":0,"num_object_copies":1904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16009,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17282","reported_seq":18074,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.068491+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:00.068491+0000","last_peered":"2024-09-15T21:27:00.068491+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:00.068491+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":752,"num_object_clones":0,"num_object_copies":1504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17439","reported_seq":24106,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.165028+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:00.165028+0000","last_peered":"2024-09-15T21:27:00.165028+0000","last_clean":"2024-09-15T21:27:00.165028+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:00.165028+0000","last_undegraded":"2024-09-15T21:27:00.165028+0000","last_fullsized":"2024-09-15T21:27:00.165028+0000","mapping_epoch":53,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3465216,"num_objects":846,"num_object_clones":0,"num_object_copies":1692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":93,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.729969+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:25:03.729969+0000","last_peered":"2024-09-15T21:25:03.729969+0000","last_clean":"2024-09-15T21:25:03.729969+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:25:03.729969+0000","last_undegraded":"2024-09-15T21:25:03.729969+0000","last_fullsized":"2024-09-15T21:25:03.729969+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17757","reported_seq":18178,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.135990+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:00.135990+0000","last_peered":"2024-09-15T21:27:00.135990+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:00.135990+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3239936,"num_objects":791,"num_object_clones":0,"num_object_copies":1582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16156,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'17891","reported_seq":18643,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.032733+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:00.032733+0000","last_peered":"2024-09-15T21:27:00.032733+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:00.032733+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730075+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:25:03.730075+0000","last_peered":"2024-09-15T21:25:03.730075+0000","last_clean":"2024-09-15T21:25:03.730075+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:25:03.730075+0000","last_undegraded":"2024-09-15T21:25:03.730075+0000","last_fullsized":"2024-09-15T21:25:03.730075+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"52'17543","reported_seq":24461,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.767088+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:26:55.767088+0000","last_peered":"2024-09-15T21:26:55.767088+0000","last_clean":"2024-09-15T21:26:55.767088+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:26:55.767088+0000","last_undegraded":"2024-09-15T21:26:55.767088+0000","last_fullsized":"2024-09-15T21:26:55.767088+0000","mapping_epoch":51,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15128,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17580","reported_seq":18248,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.895266+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:59.895266+0000","last_peered":"2024-09-15T21:26:59.895266+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:59.895266+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15227,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730447+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:25:03.730447+0000","last_peered":"2024-09-15T21:25:03.730447+0000","last_clean":"2024-09-15T21:25:03.730447+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:25:03.730447+0000","last_undegraded":"2024-09-15T21:25:03.730447+0000","last_fullsized":"2024-09-15T21:25:03.730447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.730231+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:25:03.730231+0000","last_peered":"2024-09-15T21:25:03.730231+0000","last_clean":"2024-09-15T21:25:03.730231+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:25:03.730231+0000","last_undegraded":"2024-09-15T21:25:03.730231+0000","last_fullsized":"2024-09-15T21:25:03.730231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"52'17709","reported_seq":24808,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:26:55.814183+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:26:55.814183+0000","last_peered":"2024-09-15T21:26:55.814183+0000","last_clean":"2024-09-15T21:26:55.814183+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:26:55.814183+0000","last_undegraded":"2024-09-15T21:26:55.814183+0000","last_fullsized":"2024-09-15T21:26:55.814183+0000","mapping_epoch":47,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723881,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15369,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":54456961,"num_objects":13297,"num_object_clones":0,"num_object_copies":26594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254482,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18969,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":276787200,"data_stored":276730350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432396,"kb_used_data":135536,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93939444,"statfs":{"total":96636764160,"available":96193990656,"internally_reserved":0,"allocated":138788864,"data_stored":138476867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512212,"kb_used_data":93244,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93859628,"statfs":{"total":96636764160,"available":96112259072,"internally_reserved":0,"allocated":95481856,"data_stored":95173931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626535,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482260,"kb_used_data":45184,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93889580,"statfs":{"total":96636764160,"available":96142929920,"internally_reserved":0,"allocated":46268416,"data_stored":45928745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93564928,"data_stored":93561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45355008,"data_stored":45309188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137867264,"data_stored":137859713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:01.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'17671", 'reported_seq': 18623, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:00.047948+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:00.047948+0000', 'last_peered': '2024-09-15T21:27:00.047948+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:00.047948+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7600", 'ondisk_log_start': "46'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3444736, 'num_objects': 841, 'num_object_clones': 0, 'num_object_copies': 1682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2939, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17671, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 428, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2098}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'17604", 'reported_seq': 18434, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:00.056637+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:00.056637+0000', 'last_peered': '2024-09-15T21:27:00.056637+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:00.056637+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7600", 'ondisk_log_start': "46'7600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3182592, 'num_objects': 777, 'num_object_clones': 0, 'num_object_copies': 1554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15029, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18109", 'reported_seq': 18859, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:59.987880+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:26:59.987880+0000', 'last_peered': '2024-09-15T21:26:59.987880+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:26:59.987880+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3256320, 'num_objects': 795, 'num_object_clones': 0, 'num_object_copies': 1590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15745, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'17282", 'reported_seq': 18074, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:00.068491+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:00.068491+0000', 'last_peered': '2024-09-15T21:27:00.068491+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:00.068491+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7200", 'ondisk_log_start': "46'7200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3080192, 'num_objects': 752, 'num_object_clones': 0, 'num_object_copies': 1504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14808, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2263}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'17757", 'reported_seq': 18178, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:00.135990+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:00.135990+0000', 'last_peered': '2024-09-15T21:27:00.135990+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:00.135990+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'7700", 'ondisk_log_start': "46'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3239936, 'num_objects': 791, 'num_object_clones': 0, 'num_object_copies': 1582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16156, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2415}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:01.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:01.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'17891", 'reported_seq': 18643, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:00.032733+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:00.032733+0000', 'last_peered': '2024-09-15T21:27:00.032733+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:00.032733+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3354624, 'num_objects': 819, 'num_object_clones': 0, 'num_object_copies': 1638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15198, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2311}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:01.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'17580", 'reported_seq': 18248, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:59.895266+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:26:59.895266+0000', 'last_peered': '2024-09-15T21:26:59.895266+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:26:59.895266+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7500", 'ondisk_log_start': "46'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3268608, 'num_objects': 798, 'num_object_clones': 0, 'num_object_copies': 1596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15227, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:01.802 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:02.762 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:02.763 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21596/25466 objects degraded (84.803%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":635,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12733,"data_bytes":52138644,"bytes_used":1445564416,"bytes_avail":288464728064,"bytes_total":289910292480,"degraded_objects":21596,"degraded_total":25466,"degraded_ratio":0.84803267101233015,"recovering_objects_per_sec":52,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":420,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":537},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:02.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:02.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:03.544 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:03.578 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-15T21:27:02.537779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":52138644,"num_objects":12733,"num_object_clones":0,"num_object_copies":25466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12733,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255050,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18970,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1411684,"kb_used_data":267104,"kb_used_omap":12,"kb_used_meta":1144499,"kb_avail":281703836,"statfs":{"total":289910292480,"available":288464728064,"internally_reserved":0,"allocated":273514496,"data_stored":272544308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1171966986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,7,5,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17625088,"num_objects":-4303,"num_object_clones":0,"num_object_copies":-8606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4303,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.006151"},"pg_stats":[{"pgid":"2.f","version":"54'17671","reported_seq":18623,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.047948+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:00.047948+0000","last_peered":"2024-09-15T21:27:00.047948+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:00.047948+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3444736,"num_objects":841,"num_object_clones":0,"num_object_copies":1682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17671,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":428,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2098}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17814","reported_seq":17929,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.132008+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:00.132008+0000","last_peered":"2024-09-15T21:27:00.132008+0000","last_clean":"2024-09-15T21:27:00.132008+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:00.132008+0000","last_undegraded":"2024-09-15T21:27:00.132008+0000","last_fullsized":"2024-09-15T21:27:00.132008+0000","mapping_epoch":36,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18025","reported_seq":24959,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.918786+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:00.918786+0000","last_peered":"2024-09-15T21:27:00.918786+0000","last_clean":"2024-09-15T21:27:00.918786+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:00.918786+0000","last_undegraded":"2024-09-15T21:27:00.918786+0000","last_fullsized":"2024-09-15T21:27:00.918786+0000","mapping_epoch":45,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3457048,"num_objects":845,"num_object_clones":0,"num_object_copies":1690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16748,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17799","reported_seq":17887,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.129025+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:00.129025+0000","last_peered":"2024-09-15T21:27:00.129025+0000","last_clean":"2024-09-15T21:27:00.129025+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:00.129025+0000","last_undegraded":"2024-09-15T21:27:00.129025+0000","last_fullsized":"2024-09-15T21:27:00.129025+0000","mapping_epoch":36,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3158016,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17799,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'17944","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.000745+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:00.000745+0000","last_peered":"2024-09-15T21:27:00.000745+0000","last_clean":"2024-09-15T21:27:00.000745+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:00.000745+0000","last_undegraded":"2024-09-15T21:27:00.000745+0000","last_fullsized":"2024-09-15T21:27:00.000745+0000","mapping_epoch":35,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3190784,"num_objects":779,"num_object_clones":0,"num_object_copies":1558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15802,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17604","reported_seq":18434,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.056637+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:00.056637+0000","last_peered":"2024-09-15T21:27:00.056637+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:00.056637+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3182592,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15029,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18109","reported_seq":18859,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.987880+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:26:59.987880+0000","last_peered":"2024-09-15T21:26:59.987880+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:26:59.987880+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3256320,"num_objects":795,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17623","reported_seq":17761,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.004015+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:00.004015+0000","last_peered":"2024-09-15T21:27:00.004015+0000","last_clean":"2024-09-15T21:27:00.004015+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:00.004015+0000","last_undegraded":"2024-09-15T21:27:00.004015+0000","last_fullsized":"2024-09-15T21:27:00.004015+0000","mapping_epoch":35,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15408,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17573","reported_seq":24405,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.872861+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:00.872861+0000","last_peered":"2024-09-15T21:27:00.872861+0000","last_clean":"2024-09-15T21:27:00.872861+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:00.872861+0000","last_undegraded":"2024-09-15T21:27:00.872861+0000","last_fullsized":"2024-09-15T21:27:00.872861+0000","mapping_epoch":49,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3284992,"num_objects":802,"num_object_clones":0,"num_object_copies":1604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16159,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17282","reported_seq":18074,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.068491+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:00.068491+0000","last_peered":"2024-09-15T21:27:00.068491+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:00.068491+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7200","ondisk_log_start":"46'7200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3080192,"num_objects":752,"num_object_clones":0,"num_object_copies":1504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2263}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17439","reported_seq":24106,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.165028+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:00.165028+0000","last_peered":"2024-09-15T21:27:00.165028+0000","last_clean":"2024-09-15T21:27:00.165028+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:00.165028+0000","last_undegraded":"2024-09-15T21:27:00.165028+0000","last_fullsized":"2024-09-15T21:27:00.165028+0000","mapping_epoch":53,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3465216,"num_objects":846,"num_object_clones":0,"num_object_copies":1692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17757","reported_seq":18178,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.135990+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:00.135990+0000","last_peered":"2024-09-15T21:27:00.135990+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:00.135990+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3239936,"num_objects":791,"num_object_clones":0,"num_object_copies":1582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16156,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'17891","reported_seq":18643,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:00.032733+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:00.032733+0000","last_peered":"2024-09-15T21:27:00.032733+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:00.032733+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3354624,"num_objects":819,"num_object_clones":0,"num_object_copies":1638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17674","reported_seq":24596,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.895459+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:00.895459+0000","last_peered":"2024-09-15T21:27:00.895459+0000","last_clean":"2024-09-15T21:27:00.895459+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:00.895459+0000","last_undegraded":"2024-09-15T21:27:00.895459+0000","last_fullsized":"2024-09-15T21:27:00.895459+0000","mapping_epoch":51,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3022848,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15259,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17580","reported_seq":18248,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.895266+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:26:59.895266+0000","last_peered":"2024-09-15T21:26:59.895266+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:26:59.895266+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3268608,"num_objects":798,"num_object_clones":0,"num_object_copies":1596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15227,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731021+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731021+0000","last_peered":"2024-09-15T21:26:58.731021+0000","last_clean":"2024-09-15T21:26:58.731021+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731021+0000","last_undegraded":"2024-09-15T21:26:58.731021+0000","last_fullsized":"2024-09-15T21:26:58.731021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'17849","reported_seq":24952,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:00.869654+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:00.869654+0000","last_peered":"2024-09-15T21:27:00.869654+0000","last_clean":"2024-09-15T21:27:00.869654+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:00.869654+0000","last_undegraded":"2024-09-15T21:27:00.869654+0000","last_fullsized":"2024-09-15T21:27:00.869654+0000","mapping_epoch":47,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3150441,"num_objects":770,"num_object_clones":0,"num_object_copies":1540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15509,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":52138625,"num_objects":12731,"num_object_clones":0,"num_object_copies":25462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255048,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18969,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":269889536,"data_stored":269832686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432396,"kb_used_data":135536,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93939444,"statfs":{"total":96636764160,"available":96193990656,"internally_reserved":0,"allocated":138788864,"data_stored":138476867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149817,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512212,"kb_used_data":93244,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93859628,"statfs":{"total":96636764160,"available":96112259072,"internally_reserved":0,"allocated":95481856,"data_stored":95173931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626536,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467076,"kb_used_data":38324,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93904764,"statfs":{"total":96636764160,"available":96158478336,"internally_reserved":0,"allocated":39243776,"data_stored":38893510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":439021228},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93564928,"data_stored":93561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38457344,"data_stored":38411524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137867264,"data_stored":137859713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:06.580 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:07.480 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:07.498 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-15T21:27:06.538657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42164884,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257485,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378384,"kb_used_data":246296,"kb_used_omap":12,"kb_used_meta":1132019,"kb_avail":281737136,"statfs":{"total":289910292480,"available":288498827264,"internally_reserved":0,"allocated":252207104,"data_stored":251236916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1159187466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,6,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18915328,"num_objects":-4618,"num_object_clones":0,"num_object_copies":-9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.830011"},"pg_stats":[{"pgid":"2.f","version":"54'17815","reported_seq":18933,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.025349+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:05.025349+0000","last_peered":"2024-09-15T21:27:05.025349+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:05.025349+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2854912,"num_objects":697,"num_object_clones":0,"num_object_copies":1394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17815,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17966","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.102633+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:05.102633+0000","last_peered":"2024-09-15T21:27:05.102633+0000","last_clean":"2024-09-15T21:27:05.102633+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:05.102633+0000","last_undegraded":"2024-09-15T21:27:05.102633+0000","last_fullsized":"2024-09-15T21:27:05.102633+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18186","reported_seq":25120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.976387+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:05.976387+0000","last_peered":"2024-09-15T21:27:05.976387+0000","last_clean":"2024-09-15T21:27:05.976387+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:05.976387+0000","last_undegraded":"2024-09-15T21:27:05.976387+0000","last_fullsized":"2024-09-15T21:27:05.976387+0000","mapping_epoch":45,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2797592,"num_objects":684,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16909,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17945","reported_seq":18033,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.189896+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:05.189896+0000","last_peered":"2024-09-15T21:27:05.189896+0000","last_clean":"2024-09-15T21:27:05.189896+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:05.189896+0000","last_undegraded":"2024-09-15T21:27:05.189896+0000","last_fullsized":"2024-09-15T21:27:05.189896+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560000,"num_objects":625,"num_object_clones":0,"num_object_copies":1250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18082","reported_seq":18219,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.941444+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:04.941444+0000","last_peered":"2024-09-15T21:27:04.941444+0000","last_clean":"2024-09-15T21:27:04.941444+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:04.941444+0000","last_undegraded":"2024-09-15T21:27:04.941444+0000","last_fullsized":"2024-09-15T21:27:04.941444+0000","mapping_epoch":35,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2625536,"num_objects":641,"num_object_clones":0,"num_object_copies":1282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17755","reported_seq":18587,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.067018+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:05.067018+0000","last_peered":"2024-09-15T21:27:05.067018+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:05.067018+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18250","reported_seq":19006,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.066926+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:05.066926+0000","last_peered":"2024-09-15T21:27:05.066926+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:05.066926+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2678784,"num_objects":654,"num_object_clones":0,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15886,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17769","reported_seq":17907,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.062245+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:05.062245+0000","last_peered":"2024-09-15T21:27:05.062245+0000","last_clean":"2024-09-15T21:27:05.062245+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:05.062245+0000","last_undegraded":"2024-09-15T21:27:05.062245+0000","last_fullsized":"2024-09-15T21:27:05.062245+0000","mapping_epoch":35,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2600960,"num_objects":635,"num_object_clones":0,"num_object_copies":1270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17757","reported_seq":24589,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.930782+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:05.930782+0000","last_peered":"2024-09-15T21:27:05.930782+0000","last_clean":"2024-09-15T21:27:05.930782+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:05.930782+0000","last_undegraded":"2024-09-15T21:27:05.930782+0000","last_fullsized":"2024-09-15T21:27:05.930782+0000","mapping_epoch":49,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2531328,"num_objects":618,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17432","reported_seq":18224,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.065939+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:05.065939+0000","last_peered":"2024-09-15T21:27:05.065939+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:05.065939+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2465792,"num_objects":602,"num_object_clones":0,"num_object_copies":1204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17583","reported_seq":24253,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.069482+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:05.069482+0000","last_peered":"2024-09-15T21:27:05.069482+0000","last_clean":"2024-09-15T21:27:05.069482+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:05.069482+0000","last_undegraded":"2024-09-15T21:27:05.069482+0000","last_fullsized":"2024-09-15T21:27:05.069482+0000","mapping_epoch":53,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2875392,"num_objects":702,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16134,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17905","reported_seq":18326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.198775+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:05.198775+0000","last_peered":"2024-09-15T21:27:05.198775+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:05.198775+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2633728,"num_objects":643,"num_object_clones":0,"num_object_copies":1286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16304,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18044","reported_seq":18793,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.995267+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:04.995267+0000","last_peered":"2024-09-15T21:27:04.995267+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:04.995267+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2727936,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17825","reported_seq":24747,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.929425+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:05.929425+0000","last_peered":"2024-09-15T21:27:05.929425+0000","last_clean":"2024-09-15T21:27:05.929425+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:05.929425+0000","last_undegraded":"2024-09-15T21:27:05.929425+0000","last_fullsized":"2024-09-15T21:27:05.929425+0000","mapping_epoch":51,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2404352,"num_objects":587,"num_object_clones":0,"num_object_copies":1174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15410,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17747","reported_seq":18415,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.023693+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:05.023693+0000","last_peered":"2024-09-15T21:27:05.023693+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:05.023693+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584576,"num_objects":631,"num_object_clones":0,"num_object_copies":1262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18008","reported_seq":25111,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.911880+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:05.911880+0000","last_peered":"2024-09-15T21:27:05.911880+0000","last_clean":"2024-09-15T21:27:05.911880+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:05.911880+0000","last_undegraded":"2024-09-15T21:27:05.911880+0000","last_fullsized":"2024-09-15T21:27:05.911880+0000","mapping_epoch":47,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2499177,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42164865,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257483,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19053,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249946112,"data_stored":249889262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415104,"kb_used_data":130732,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93956736,"statfs":{"total":96636764160,"available":96211697664,"internally_reserved":0,"allocated":133869568,"data_stored":133557571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495336,"kb_used_data":84692,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93876504,"statfs":{"total":96636764160,"available":96129540096,"internally_reserved":0,"allocated":86724608,"data_stored":86416683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":420474545},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467944,"kb_used_data":30872,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93903896,"statfs":{"total":96636764160,"available":96157589504,"internally_reserved":0,"allocated":31612928,"data_stored":31262662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86196224,"data_stored":86192745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30789632,"data_stored":30743812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132960256,"data_stored":132952705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:07.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:08.101 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:08.139 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-15T21:27:06.538657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42164884,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257485,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378384,"kb_used_data":246296,"kb_used_omap":12,"kb_used_meta":1132019,"kb_avail":281737136,"statfs":{"total":289910292480,"available":288498827264,"internally_reserved":0,"allocated":252207104,"data_stored":251236916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1159187466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,6,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18915328,"num_objects":-4618,"num_object_clones":0,"num_object_copies":-9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.830011"},"pg_stats":[{"pgid":"2.f","version":"54'17815","reported_seq":18933,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.025349+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:05.025349+0000","last_peered":"2024-09-15T21:27:05.025349+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:05.025349+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2854912,"num_objects":697,"num_object_clones":0,"num_object_copies":1394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17815,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17966","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.102633+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:05.102633+0000","last_peered":"2024-09-15T21:27:05.102633+0000","last_clean":"2024-09-15T21:27:05.102633+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:05.102633+0000","last_undegraded":"2024-09-15T21:27:05.102633+0000","last_fullsized":"2024-09-15T21:27:05.102633+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18186","reported_seq":25120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.976387+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:05.976387+0000","last_peered":"2024-09-15T21:27:05.976387+0000","last_clean":"2024-09-15T21:27:05.976387+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:05.976387+0000","last_undegraded":"2024-09-15T21:27:05.976387+0000","last_fullsized":"2024-09-15T21:27:05.976387+0000","mapping_epoch":45,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2797592,"num_objects":684,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16909,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17945","reported_seq":18033,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.189896+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:05.189896+0000","last_peered":"2024-09-15T21:27:05.189896+0000","last_clean":"2024-09-15T21:27:05.189896+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:05.189896+0000","last_undegraded":"2024-09-15T21:27:05.189896+0000","last_fullsized":"2024-09-15T21:27:05.189896+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560000,"num_objects":625,"num_object_clones":0,"num_object_copies":1250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18082","reported_seq":18219,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.941444+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:04.941444+0000","last_peered":"2024-09-15T21:27:04.941444+0000","last_clean":"2024-09-15T21:27:04.941444+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:04.941444+0000","last_undegraded":"2024-09-15T21:27:04.941444+0000","last_fullsized":"2024-09-15T21:27:04.941444+0000","mapping_epoch":35,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2625536,"num_objects":641,"num_object_clones":0,"num_object_copies":1282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17755","reported_seq":18587,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.067018+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:05.067018+0000","last_peered":"2024-09-15T21:27:05.067018+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:05.067018+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18250","reported_seq":19006,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.066926+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:05.066926+0000","last_peered":"2024-09-15T21:27:05.066926+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:05.066926+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2678784,"num_objects":654,"num_object_clones":0,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15886,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17769","reported_seq":17907,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.062245+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:05.062245+0000","last_peered":"2024-09-15T21:27:05.062245+0000","last_clean":"2024-09-15T21:27:05.062245+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:05.062245+0000","last_undegraded":"2024-09-15T21:27:05.062245+0000","last_fullsized":"2024-09-15T21:27:05.062245+0000","mapping_epoch":35,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2600960,"num_objects":635,"num_object_clones":0,"num_object_copies":1270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17757","reported_seq":24589,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.930782+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:05.930782+0000","last_peered":"2024-09-15T21:27:05.930782+0000","last_clean":"2024-09-15T21:27:05.930782+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:05.930782+0000","last_undegraded":"2024-09-15T21:27:05.930782+0000","last_fullsized":"2024-09-15T21:27:05.930782+0000","mapping_epoch":49,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2531328,"num_objects":618,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17432","reported_seq":18224,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.065939+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:05.065939+0000","last_peered":"2024-09-15T21:27:05.065939+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:05.065939+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2465792,"num_objects":602,"num_object_clones":0,"num_object_copies":1204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17583","reported_seq":24253,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.069482+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:05.069482+0000","last_peered":"2024-09-15T21:27:05.069482+0000","last_clean":"2024-09-15T21:27:05.069482+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:05.069482+0000","last_undegraded":"2024-09-15T21:27:05.069482+0000","last_fullsized":"2024-09-15T21:27:05.069482+0000","mapping_epoch":53,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2875392,"num_objects":702,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16134,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17905","reported_seq":18326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.198775+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:05.198775+0000","last_peered":"2024-09-15T21:27:05.198775+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:05.198775+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2633728,"num_objects":643,"num_object_clones":0,"num_object_copies":1286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16304,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18044","reported_seq":18793,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.995267+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:04.995267+0000","last_peered":"2024-09-15T21:27:04.995267+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:04.995267+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2727936,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17825","reported_seq":24747,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.929425+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:05.929425+0000","last_peered":"2024-09-15T21:27:05.929425+0000","last_clean":"2024-09-15T21:27:05.929425+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:05.929425+0000","last_undegraded":"2024-09-15T21:27:05.929425+0000","last_fullsized":"2024-09-15T21:27:05.929425+0000","mapping_epoch":51,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2404352,"num_objects":587,"num_object_clones":0,"num_object_copies":1174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15410,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17747","reported_seq":18415,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.023693+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:05.023693+0000","last_peered":"2024-09-15T21:27:05.023693+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:05.023693+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584576,"num_objects":631,"num_object_clones":0,"num_object_copies":1262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18008","reported_seq":25111,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.911880+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:05.911880+0000","last_peered":"2024-09-15T21:27:05.911880+0000","last_clean":"2024-09-15T21:27:05.911880+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:05.911880+0000","last_undegraded":"2024-09-15T21:27:05.911880+0000","last_fullsized":"2024-09-15T21:27:05.911880+0000","mapping_epoch":47,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2499177,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42164865,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257483,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19053,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249946112,"data_stored":249889262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415104,"kb_used_data":130732,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93956736,"statfs":{"total":96636764160,"available":96211697664,"internally_reserved":0,"allocated":133869568,"data_stored":133557571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495336,"kb_used_data":84692,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93876504,"statfs":{"total":96636764160,"available":96129540096,"internally_reserved":0,"allocated":86724608,"data_stored":86416683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":420474545},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467944,"kb_used_data":30872,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93903896,"statfs":{"total":96636764160,"available":96157589504,"internally_reserved":0,"allocated":31612928,"data_stored":31262662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86196224,"data_stored":86192745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30789632,"data_stored":30743812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132960256,"data_stored":132952705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'17815", 'reported_seq': 18933, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.025349+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:05.025349+0000', 'last_peered': '2024-09-15T21:27:05.025349+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:05.025349+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2854912, 'num_objects': 697, 'num_object_clones': 0, 'num_object_copies': 1394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2855, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17815, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 512, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2158}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'17755", 'reported_seq': 18587, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.067018+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:05.067018+0000', 'last_peered': '2024-09-15T21:27:05.067018+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:05.067018+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7700", 'ondisk_log_start': "46'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2564096, 'num_objects': 626, 'num_object_clones': 0, 'num_object_copies': 1252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15180, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18250", 'reported_seq': 19006, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.066926+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:05.066926+0000', 'last_peered': '2024-09-15T21:27:05.066926+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:05.066926+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2678784, 'num_objects': 654, 'num_object_clones': 0, 'num_object_copies': 1308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15886, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'17432", 'reported_seq': 18224, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.065939+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:05.065939+0000', 'last_peered': '2024-09-15T21:27:05.065939+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:05.065939+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7400", 'ondisk_log_start': "46'7400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2465792, 'num_objects': 602, 'num_object_clones': 0, 'num_object_copies': 1204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14958, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2413}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'17905", 'reported_seq': 18326, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.198775+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:05.198775+0000', 'last_peered': '2024-09-15T21:27:05.198775+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:05.198775+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'7900", 'ondisk_log_start': "46'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2633728, 'num_objects': 643, 'num_object_clones': 0, 'num_object_copies': 1286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16304, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2563}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18044", 'reported_seq': 18793, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:04.995267+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:04.995267+0000', 'last_peered': '2024-09-15T21:27:04.995267+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:04.995267+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2727936, 'num_objects': 666, 'num_object_clones': 0, 'num_object_copies': 1332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15351, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2464}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:08.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'17747", 'reported_seq': 18415, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:05.023693+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:05.023693+0000', 'last_peered': '2024-09-15T21:27:05.023693+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:05.023693+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7700", 'ondisk_log_start': "46'7700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2584576, 'num_objects': 631, 'num_object_clones': 0, 'num_object_copies': 1262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15394, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:08.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:08.573 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21512/20596 objects degraded (104.447%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":10298,"data_bytes":42164884,"bytes_used":1411465216,"bytes_avail":288498827264,"bytes_total":289910292480,"degraded_objects":21512,"degraded_total":20596,"degraded_ratio":1.0444746552728685,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":522},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:08.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:08.592 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:09.189 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:09.245 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-15T21:27:08.539141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":42164884,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257485,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1378384,"kb_used_data":246296,"kb_used_omap":12,"kb_used_meta":1132019,"kb_avail":281737136,"statfs":{"total":289910292480,"available":288498827264,"internally_reserved":0,"allocated":252207104,"data_stored":251236916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1159187466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,6,6,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18915328,"num_objects":-4618,"num_object_clones":0,"num_object_copies":-9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.006559"},"pg_stats":[{"pgid":"2.f","version":"54'17815","reported_seq":18933,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.025349+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:05.025349+0000","last_peered":"2024-09-15T21:27:05.025349+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:05.025349+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2854912,"num_objects":697,"num_object_clones":0,"num_object_copies":1394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17815,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'17966","reported_seq":18081,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.102633+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:05.102633+0000","last_peered":"2024-09-15T21:27:05.102633+0000","last_clean":"2024-09-15T21:27:05.102633+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:05.102633+0000","last_undegraded":"2024-09-15T21:27:05.102633+0000","last_fullsized":"2024-09-15T21:27:05.102633+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2760704,"num_objects":674,"num_object_clones":0,"num_object_copies":1348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18186","reported_seq":25120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.976387+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:05.976387+0000","last_peered":"2024-09-15T21:27:05.976387+0000","last_clean":"2024-09-15T21:27:05.976387+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:05.976387+0000","last_undegraded":"2024-09-15T21:27:05.976387+0000","last_fullsized":"2024-09-15T21:27:05.976387+0000","mapping_epoch":45,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2797592,"num_objects":684,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":684,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16909,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'17945","reported_seq":18033,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.189896+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:05.189896+0000","last_peered":"2024-09-15T21:27:05.189896+0000","last_clean":"2024-09-15T21:27:05.189896+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:05.189896+0000","last_undegraded":"2024-09-15T21:27:05.189896+0000","last_fullsized":"2024-09-15T21:27:05.189896+0000","mapping_epoch":36,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2560000,"num_objects":625,"num_object_clones":0,"num_object_copies":1250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17945,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18082","reported_seq":18219,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.941444+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:04.941444+0000","last_peered":"2024-09-15T21:27:04.941444+0000","last_clean":"2024-09-15T21:27:04.941444+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:04.941444+0000","last_undegraded":"2024-09-15T21:27:04.941444+0000","last_fullsized":"2024-09-15T21:27:04.941444+0000","mapping_epoch":35,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2625536,"num_objects":641,"num_object_clones":0,"num_object_copies":1282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15940,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17755","reported_seq":18587,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.067018+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:05.067018+0000","last_peered":"2024-09-15T21:27:05.067018+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:05.067018+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15180,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18250","reported_seq":19006,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.066926+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:05.066926+0000","last_peered":"2024-09-15T21:27:05.066926+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:05.066926+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2678784,"num_objects":654,"num_object_clones":0,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15886,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17769","reported_seq":17907,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.062245+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:05.062245+0000","last_peered":"2024-09-15T21:27:05.062245+0000","last_clean":"2024-09-15T21:27:05.062245+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:05.062245+0000","last_undegraded":"2024-09-15T21:27:05.062245+0000","last_fullsized":"2024-09-15T21:27:05.062245+0000","mapping_epoch":35,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2600960,"num_objects":635,"num_object_clones":0,"num_object_copies":1270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17757","reported_seq":24589,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.930782+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:05.930782+0000","last_peered":"2024-09-15T21:27:05.930782+0000","last_clean":"2024-09-15T21:27:05.930782+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:05.930782+0000","last_undegraded":"2024-09-15T21:27:05.930782+0000","last_fullsized":"2024-09-15T21:27:05.930782+0000","mapping_epoch":49,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2531328,"num_objects":618,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17432","reported_seq":18224,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.065939+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:05.065939+0000","last_peered":"2024-09-15T21:27:05.065939+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:05.065939+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7400","ondisk_log_start":"46'7400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2465792,"num_objects":602,"num_object_clones":0,"num_object_copies":1204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17583","reported_seq":24253,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.069482+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:05.069482+0000","last_peered":"2024-09-15T21:27:05.069482+0000","last_clean":"2024-09-15T21:27:05.069482+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:05.069482+0000","last_undegraded":"2024-09-15T21:27:05.069482+0000","last_fullsized":"2024-09-15T21:27:05.069482+0000","mapping_epoch":53,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2875392,"num_objects":702,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16134,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'17905","reported_seq":18326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.198775+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:05.198775+0000","last_peered":"2024-09-15T21:27:05.198775+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:05.198775+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2633728,"num_objects":643,"num_object_clones":0,"num_object_copies":1286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16304,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18044","reported_seq":18793,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.995267+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:04.995267+0000","last_peered":"2024-09-15T21:27:04.995267+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:04.995267+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2727936,"num_objects":666,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17825","reported_seq":24747,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.929425+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:05.929425+0000","last_peered":"2024-09-15T21:27:05.929425+0000","last_clean":"2024-09-15T21:27:05.929425+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:05.929425+0000","last_undegraded":"2024-09-15T21:27:05.929425+0000","last_fullsized":"2024-09-15T21:27:05.929425+0000","mapping_epoch":51,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2404352,"num_objects":587,"num_object_clones":0,"num_object_copies":1174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15410,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17747","reported_seq":18415,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:05.023693+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:05.023693+0000","last_peered":"2024-09-15T21:27:05.023693+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:05.023693+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2584576,"num_objects":631,"num_object_clones":0,"num_object_copies":1262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18008","reported_seq":25111,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:05.911880+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:05.911880+0000","last_peered":"2024-09-15T21:27:05.911880+0000","last_clean":"2024-09-15T21:27:05.911880+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:05.911880+0000","last_undegraded":"2024-09-15T21:27:05.911880+0000","last_fullsized":"2024-09-15T21:27:05.911880+0000","mapping_epoch":47,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2499177,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":42164865,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257483,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19053,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249946112,"data_stored":249889262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415104,"kb_used_data":130732,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93956736,"statfs":{"total":96636764160,"available":96211697664,"internally_reserved":0,"allocated":133869568,"data_stored":133557571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149818,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495336,"kb_used_data":84692,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93876504,"statfs":{"total":96636764160,"available":96129540096,"internally_reserved":0,"allocated":86724608,"data_stored":86416683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":420474545},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467944,"kb_used_data":30872,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93903896,"statfs":{"total":96636764160,"available":96157589504,"internally_reserved":0,"allocated":31612928,"data_stored":31262662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":447540908},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86196224,"data_stored":86192745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30789632,"data_stored":30743812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132960256,"data_stored":132952705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:12.247 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:13.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:13.416 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:13.818 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-15T21:27:12.540077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33354388,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259636,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19167,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1375156,"kb_used_data":229188,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281740364,"statfs":{"total":289910292480,"available":288502132736,"internally_reserved":0,"allocated":234688512,"data_stored":233718324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1173408778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21102592,"num_objects":-5152,"num_object_clones":0,"num_object_copies":-10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007156"},"pg_stats":[{"pgid":"2.f","version":"54'17949","reported_seq":19295,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.136468+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:10.136468+0000","last_peered":"2024-09-15T21:27:10.136468+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:10.136468+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17949,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18113","reported_seq":18228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.209120+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:10.209120+0000","last_peered":"2024-09-15T21:27:10.209120+0000","last_clean":"2024-09-15T21:27:10.209120+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:10.209120+0000","last_undegraded":"2024-09-15T21:27:10.209120+0000","last_fullsized":"2024-09-15T21:27:10.209120+0000","mapping_epoch":36,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158592,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16839,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18324","reported_seq":25258,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.908169+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:10.908169+0000","last_peered":"2024-09-15T21:27:10.908169+0000","last_clean":"2024-09-15T21:27:10.908169+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:10.908169+0000","last_undegraded":"2024-09-15T21:27:10.908169+0000","last_fullsized":"2024-09-15T21:27:10.908169+0000","mapping_epoch":45,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232344,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17047,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18081","reported_seq":18169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.191524+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:10.191524+0000","last_peered":"2024-09-15T21:27:10.191524+0000","last_clean":"2024-09-15T21:27:10.191524+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:10.191524+0000","last_undegraded":"2024-09-15T21:27:10.191524+0000","last_fullsized":"2024-09-15T21:27:10.191524+0000","mapping_epoch":36,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2002944,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18081,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18227","reported_seq":18364,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.106501+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:10.106501+0000","last_peered":"2024-09-15T21:27:10.106501+0000","last_clean":"2024-09-15T21:27:10.106501+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:10.106501+0000","last_undegraded":"2024-09-15T21:27:10.106501+0000","last_fullsized":"2024-09-15T21:27:10.106501+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16085,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17880","reported_seq":18712,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.154218+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:10.154218+0000","last_peered":"2024-09-15T21:27:10.154218+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:10.154218+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052096,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15305,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18394","reported_seq":19150,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.140713+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:10.140713+0000","last_peered":"2024-09-15T21:27:10.140713+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:10.140713+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16030,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17894","reported_seq":18032,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.073172+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:10.073172+0000","last_peered":"2024-09-15T21:27:10.073172+0000","last_clean":"2024-09-15T21:27:10.073172+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:10.073172+0000","last_undegraded":"2024-09-15T21:27:10.073172+0000","last_fullsized":"2024-09-15T21:27:10.073172+0000","mapping_epoch":35,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17871","reported_seq":24703,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.990126+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:10.990126+0000","last_peered":"2024-09-15T21:27:10.990126+0000","last_clean":"2024-09-15T21:27:10.990126+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:10.990126+0000","last_undegraded":"2024-09-15T21:27:10.990126+0000","last_fullsized":"2024-09-15T21:27:10.990126+0000","mapping_epoch":49,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064384,"num_objects":504,"num_object_clones":0,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16457,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17566","reported_seq":18359,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.138796+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:10.138796+0000","last_peered":"2024-09-15T21:27:10.138796+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:10.138796+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916928,"num_objects":468,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17724","reported_seq":24394,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.183937+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:10.183937+0000","last_peered":"2024-09-15T21:27:10.183937+0000","last_clean":"2024-09-15T21:27:10.183937+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:10.183937+0000","last_undegraded":"2024-09-15T21:27:10.183937+0000","last_fullsized":"2024-09-15T21:27:10.183937+0000","mapping_epoch":53,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18050","reported_seq":18471,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.135808+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:10.135808+0000","last_peered":"2024-09-15T21:27:10.135808+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:10.135808+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2039808,"num_objects":498,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16449,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18165","reported_seq":18918,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.104451+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:10.104451+0000","last_peered":"2024-09-15T21:27:10.104451+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:10.104451+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232320,"num_objects":545,"num_object_clones":0,"num_object_copies":1090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17961","reported_seq":24883,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:11.005997+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:11.005997+0000","last_peered":"2024-09-15T21:27:11.005997+0000","last_clean":"2024-09-15T21:27:11.005997+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:11.005997+0000","last_undegraded":"2024-09-15T21:27:11.005997+0000","last_fullsized":"2024-09-15T21:27:11.005997+0000","mapping_epoch":51,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1847296,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17891","reported_seq":18559,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.095054+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:10.095054+0000","last_peered":"2024-09-15T21:27:10.095054+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:10.095054+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994752,"num_objects":487,"num_object_clones":0,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18130","reported_seq":25233,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.998559+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:10.998559+0000","last_peered":"2024-09-15T21:27:10.998559+0000","last_clean":"2024-09-15T21:27:10.998559+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:10.998559+0000","last_undegraded":"2024-09-15T21:27:10.998559+0000","last_fullsized":"2024-09-15T21:27:10.998559+0000","mapping_epoch":47,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999465,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15790,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33354369,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259634,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19166,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":232198144,"data_stored":232141294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432940,"kb_used_data":126364,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93938900,"statfs":{"total":96636764160,"available":96193433600,"internally_reserved":0,"allocated":129396736,"data_stored":129084739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":78464,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":80347136,"data_stored":80039211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448952,"kb_used_data":24360,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93922888,"statfs":{"total":96636764160,"available":96177037312,"internally_reserved":0,"allocated":24944640,"data_stored":24594374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79609856,"data_stored":79606377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24141824,"data_stored":24096004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128446464,"data_stored":128438913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:13.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:14.454 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:14.454 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:14.475 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-15T21:27:12.540077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33354388,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259636,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19167,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1375156,"kb_used_data":229188,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281740364,"statfs":{"total":289910292480,"available":288502132736,"internally_reserved":0,"allocated":234688512,"data_stored":233718324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1173408778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21102592,"num_objects":-5152,"num_object_clones":0,"num_object_copies":-10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007156"},"pg_stats":[{"pgid":"2.f","version":"54'17949","reported_seq":19295,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.136468+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:10.136468+0000","last_peered":"2024-09-15T21:27:10.136468+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:10.136468+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17949,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18113","reported_seq":18228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.209120+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:10.209120+0000","last_peered":"2024-09-15T21:27:10.209120+0000","last_clean":"2024-09-15T21:27:10.209120+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:10.209120+0000","last_undegraded":"2024-09-15T21:27:10.209120+0000","last_fullsized":"2024-09-15T21:27:10.209120+0000","mapping_epoch":36,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158592,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16839,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18324","reported_seq":25258,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.908169+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:10.908169+0000","last_peered":"2024-09-15T21:27:10.908169+0000","last_clean":"2024-09-15T21:27:10.908169+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:10.908169+0000","last_undegraded":"2024-09-15T21:27:10.908169+0000","last_fullsized":"2024-09-15T21:27:10.908169+0000","mapping_epoch":45,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232344,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17047,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18081","reported_seq":18169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.191524+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:10.191524+0000","last_peered":"2024-09-15T21:27:10.191524+0000","last_clean":"2024-09-15T21:27:10.191524+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:10.191524+0000","last_undegraded":"2024-09-15T21:27:10.191524+0000","last_fullsized":"2024-09-15T21:27:10.191524+0000","mapping_epoch":36,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2002944,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18081,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18227","reported_seq":18364,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.106501+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:10.106501+0000","last_peered":"2024-09-15T21:27:10.106501+0000","last_clean":"2024-09-15T21:27:10.106501+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:10.106501+0000","last_undegraded":"2024-09-15T21:27:10.106501+0000","last_fullsized":"2024-09-15T21:27:10.106501+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16085,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17880","reported_seq":18712,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.154218+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:10.154218+0000","last_peered":"2024-09-15T21:27:10.154218+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:10.154218+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052096,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15305,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18394","reported_seq":19150,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.140713+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:10.140713+0000","last_peered":"2024-09-15T21:27:10.140713+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:10.140713+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16030,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17894","reported_seq":18032,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.073172+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:10.073172+0000","last_peered":"2024-09-15T21:27:10.073172+0000","last_clean":"2024-09-15T21:27:10.073172+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:10.073172+0000","last_undegraded":"2024-09-15T21:27:10.073172+0000","last_fullsized":"2024-09-15T21:27:10.073172+0000","mapping_epoch":35,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17871","reported_seq":24703,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.990126+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:10.990126+0000","last_peered":"2024-09-15T21:27:10.990126+0000","last_clean":"2024-09-15T21:27:10.990126+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:10.990126+0000","last_undegraded":"2024-09-15T21:27:10.990126+0000","last_fullsized":"2024-09-15T21:27:10.990126+0000","mapping_epoch":49,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064384,"num_objects":504,"num_object_clones":0,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16457,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17566","reported_seq":18359,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.138796+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:10.138796+0000","last_peered":"2024-09-15T21:27:10.138796+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:10.138796+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916928,"num_objects":468,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17724","reported_seq":24394,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.183937+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:10.183937+0000","last_peered":"2024-09-15T21:27:10.183937+0000","last_clean":"2024-09-15T21:27:10.183937+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:10.183937+0000","last_undegraded":"2024-09-15T21:27:10.183937+0000","last_fullsized":"2024-09-15T21:27:10.183937+0000","mapping_epoch":53,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18050","reported_seq":18471,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.135808+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:10.135808+0000","last_peered":"2024-09-15T21:27:10.135808+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:10.135808+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2039808,"num_objects":498,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16449,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18165","reported_seq":18918,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.104451+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:10.104451+0000","last_peered":"2024-09-15T21:27:10.104451+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:10.104451+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232320,"num_objects":545,"num_object_clones":0,"num_object_copies":1090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17961","reported_seq":24883,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:11.005997+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:11.005997+0000","last_peered":"2024-09-15T21:27:11.005997+0000","last_clean":"2024-09-15T21:27:11.005997+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:11.005997+0000","last_undegraded":"2024-09-15T21:27:11.005997+0000","last_fullsized":"2024-09-15T21:27:11.005997+0000","mapping_epoch":51,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1847296,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17891","reported_seq":18559,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.095054+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:10.095054+0000","last_peered":"2024-09-15T21:27:10.095054+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:10.095054+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994752,"num_objects":487,"num_object_clones":0,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18130","reported_seq":25233,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.998559+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:10.998559+0000","last_peered":"2024-09-15T21:27:10.998559+0000","last_clean":"2024-09-15T21:27:10.998559+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:10.998559+0000","last_undegraded":"2024-09-15T21:27:10.998559+0000","last_fullsized":"2024-09-15T21:27:10.998559+0000","mapping_epoch":47,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999465,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15790,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33354369,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259634,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19166,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":232198144,"data_stored":232141294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432940,"kb_used_data":126364,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93938900,"statfs":{"total":96636764160,"available":96193433600,"internally_reserved":0,"allocated":129396736,"data_stored":129084739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":78464,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":80347136,"data_stored":80039211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448952,"kb_used_data":24360,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93922888,"statfs":{"total":96636764160,"available":96177037312,"internally_reserved":0,"allocated":24944640,"data_stored":24594374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79609856,"data_stored":79606377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24141824,"data_stored":24096004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128446464,"data_stored":128438913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'17949", 'reported_seq': 19295, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.136468+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:10.136468+0000', 'last_peered': '2024-09-15T21:27:10.136468+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:10.136468+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'7900", 'ondisk_log_start': "46'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2306048, 'num_objects': 563, 'num_object_clones': 0, 'num_object_copies': 1126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2742, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17949, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 625, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2179}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'17880", 'reported_seq': 18712, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.154218+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:10.154218+0000', 'last_peered': '2024-09-15T21:27:10.154218+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:10.154218+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2052096, 'num_objects': 501, 'num_object_clones': 0, 'num_object_copies': 1002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15305, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:14.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18394", 'reported_seq': 19150, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.140713+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:10.140713+0000', 'last_peered': '2024-09-15T21:27:10.140713+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:10.140713+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2088960, 'num_objects': 510, 'num_object_clones': 0, 'num_object_copies': 1020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16030, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2654}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'17566", 'reported_seq': 18359, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.138796+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:10.138796+0000', 'last_peered': '2024-09-15T21:27:10.138796+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:10.138796+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7500", 'ondisk_log_start': "46'7500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1916928, 'num_objects': 468, 'num_object_clones': 0, 'num_object_copies': 936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15092, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2547}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18050", 'reported_seq': 18471, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.135808+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:10.135808+0000', 'last_peered': '2024-09-15T21:27:10.135808+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:10.135808+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2039808, 'num_objects': 498, 'num_object_clones': 0, 'num_object_copies': 996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16449, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2708}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18165", 'reported_seq': 18918, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.104451+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:10.104451+0000', 'last_peered': '2024-09-15T21:27:10.104451+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:10.104451+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2232320, 'num_objects': 545, 'num_object_clones': 0, 'num_object_copies': 1090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15472, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:14.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'17891", 'reported_seq': 18559, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:10.095054+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:10.095054+0000', 'last_peered': '2024-09-15T21:27:10.095054+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:10.095054+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1994752, 'num_objects': 487, 'num_object_clones': 0, 'num_object_copies': 974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15538, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.478 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:15.154 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:15.154 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21399/16294 objects degraded (131.331%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":648,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":8147,"data_bytes":33354388,"bytes_used":1408159744,"bytes_avail":288502132736,"bytes_total":289910292480,"degraded_objects":21399,"degraded_total":16294,"degraded_ratio":1.313305511231128,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:15.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:15.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:15.662 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:15.663 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:15.721 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-15T21:27:14.540500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33354388,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259636,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19167,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1375156,"kb_used_data":229188,"kb_used_omap":12,"kb_used_meta":1145907,"kb_avail":281740364,"statfs":{"total":289910292480,"available":288502132736,"internally_reserved":0,"allocated":234688512,"data_stored":233718324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1173408778},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18784256,"num_objects":-4586,"num_object_clones":0,"num_object_copies":-9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002720"},"pg_stats":[{"pgid":"2.f","version":"54'17949","reported_seq":19295,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.136468+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:10.136468+0000","last_peered":"2024-09-15T21:27:10.136468+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:10.136468+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17949,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18113","reported_seq":18228,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.209120+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:10.209120+0000","last_peered":"2024-09-15T21:27:10.209120+0000","last_clean":"2024-09-15T21:27:10.209120+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:10.209120+0000","last_undegraded":"2024-09-15T21:27:10.209120+0000","last_fullsized":"2024-09-15T21:27:10.209120+0000","mapping_epoch":36,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158592,"num_objects":527,"num_object_clones":0,"num_object_copies":1054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16839,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18324","reported_seq":25258,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.908169+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:10.908169+0000","last_peered":"2024-09-15T21:27:10.908169+0000","last_clean":"2024-09-15T21:27:10.908169+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:10.908169+0000","last_undegraded":"2024-09-15T21:27:10.908169+0000","last_fullsized":"2024-09-15T21:27:10.908169+0000","mapping_epoch":45,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232344,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17047,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18081","reported_seq":18169,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.191524+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:10.191524+0000","last_peered":"2024-09-15T21:27:10.191524+0000","last_clean":"2024-09-15T21:27:10.191524+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:10.191524+0000","last_undegraded":"2024-09-15T21:27:10.191524+0000","last_fullsized":"2024-09-15T21:27:10.191524+0000","mapping_epoch":36,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2002944,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18081,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18227","reported_seq":18364,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.106501+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:10.106501+0000","last_peered":"2024-09-15T21:27:10.106501+0000","last_clean":"2024-09-15T21:27:10.106501+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:10.106501+0000","last_undegraded":"2024-09-15T21:27:10.106501+0000","last_fullsized":"2024-09-15T21:27:10.106501+0000","mapping_epoch":35,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16085,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17880","reported_seq":18712,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.154218+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:10.154218+0000","last_peered":"2024-09-15T21:27:10.154218+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:10.154218+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2052096,"num_objects":501,"num_object_clones":0,"num_object_copies":1002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15305,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18394","reported_seq":19150,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.140713+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:10.140713+0000","last_peered":"2024-09-15T21:27:10.140713+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:10.140713+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16030,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2654}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'17894","reported_seq":18032,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.073172+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:10.073172+0000","last_peered":"2024-09-15T21:27:10.073172+0000","last_clean":"2024-09-15T21:27:10.073172+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:10.073172+0000","last_undegraded":"2024-09-15T21:27:10.073172+0000","last_fullsized":"2024-09-15T21:27:10.073172+0000","mapping_epoch":35,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2088960,"num_objects":510,"num_object_clones":0,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'17871","reported_seq":24703,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.990126+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:10.990126+0000","last_peered":"2024-09-15T21:27:10.990126+0000","last_clean":"2024-09-15T21:27:10.990126+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:10.990126+0000","last_undegraded":"2024-09-15T21:27:10.990126+0000","last_fullsized":"2024-09-15T21:27:10.990126+0000","mapping_epoch":49,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064384,"num_objects":504,"num_object_clones":0,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16457,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17566","reported_seq":18359,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.138796+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:10.138796+0000","last_peered":"2024-09-15T21:27:10.138796+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:10.138796+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7500","ondisk_log_start":"46'7500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916928,"num_objects":468,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17724","reported_seq":24394,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.183937+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:10.183937+0000","last_peered":"2024-09-15T21:27:10.183937+0000","last_clean":"2024-09-15T21:27:10.183937+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:10.183937+0000","last_undegraded":"2024-09-15T21:27:10.183937+0000","last_fullsized":"2024-09-15T21:27:10.183937+0000","mapping_epoch":53,"log_start":"46'7700","ondisk_log_start":"46'7700","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2297856,"num_objects":561,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18050","reported_seq":18471,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.135808+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:10.135808+0000","last_peered":"2024-09-15T21:27:10.135808+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:10.135808+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2039808,"num_objects":498,"num_object_clones":0,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16449,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2708}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18165","reported_seq":18918,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.104451+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:10.104451+0000","last_peered":"2024-09-15T21:27:10.104451+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:10.104451+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232320,"num_objects":545,"num_object_clones":0,"num_object_copies":1090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2585}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'17961","reported_seq":24883,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:11.005997+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:11.005997+0000","last_peered":"2024-09-15T21:27:11.005997+0000","last_clean":"2024-09-15T21:27:11.005997+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:11.005997+0000","last_undegraded":"2024-09-15T21:27:11.005997+0000","last_fullsized":"2024-09-15T21:27:11.005997+0000","mapping_epoch":51,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1847296,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'17891","reported_seq":18559,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:10.095054+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:10.095054+0000","last_peered":"2024-09-15T21:27:10.095054+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:10.095054+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1994752,"num_objects":487,"num_object_clones":0,"num_object_copies":974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15538,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18130","reported_seq":25233,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:10.998559+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:10.998559+0000","last_peered":"2024-09-15T21:27:10.998559+0000","last_clean":"2024-09-15T21:27:10.998559+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:10.998559+0000","last_undegraded":"2024-09-15T21:27:10.998559+0000","last_fullsized":"2024-09-15T21:27:10.998559+0000","mapping_epoch":47,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999465,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15790,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":33354369,"num_objects":8145,"num_object_clones":0,"num_object_copies":16290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259634,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19166,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":232198144,"data_stored":232141294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953593,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432940,"kb_used_data":126364,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93938900,"statfs":{"total":96636764160,"available":96193433600,"internally_reserved":0,"allocated":129396736,"data_stored":129084739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149819,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":78464,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":80347136,"data_stored":80039211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":424734385},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448952,"kb_used_data":24360,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93922888,"statfs":{"total":96636764160,"available":96177037312,"internally_reserved":0,"allocated":24944640,"data_stored":24594374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":434761388},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79609856,"data_stored":79606377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24141824,"data_stored":24096004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128446464,"data_stored":128438913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:18.724 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:19.980 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:19.982 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:20.092 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-15T21:27:18.541311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24531604,"num_objects":5993,"num_object_clones":0,"num_object_copies":11986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5993,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261790,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19297,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160877,"ondisk_log_size":160877,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1392748,"kb_used_data":212088,"kb_used_omap":12,"kb_used_meta":1180595,"kb_avail":281722772,"statfs":{"total":289910292480,"available":288484118528,"internally_reserved":0,"allocated":217178112,"data_stored":216207924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1208929290},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,1,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17633280,"num_objects":-4305,"num_object_clones":0,"num_object_copies":-8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002654"},"pg_stats":[{"pgid":"2.f","version":"54'18073","reported_seq":19679,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.162823+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:15.162823+0000","last_peered":"2024-09-15T21:27:15.162823+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:15.162823+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1798144,"num_objects":439,"num_object_clones":0,"num_object_copies":878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18073,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":755,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2173}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18246","reported_seq":18361,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:15.137524+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:15.137524+0000","last_peered":"2024-09-15T21:27:15.137524+0000","last_clean":"2024-09-15T21:27:15.137524+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:15.137524+0000","last_undegraded":"2024-09-15T21:27:15.137524+0000","last_fullsized":"2024-09-15T21:27:15.137524+0000","mapping_epoch":36,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613824,"num_objects":394,"num_object_clones":0,"num_object_copies":788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16972,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18464","reported_seq":25398,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007276+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:16.007276+0000","last_peered":"2024-09-15T21:27:16.007276+0000","last_clean":"2024-09-15T21:27:16.007276+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:16.007276+0000","last_undegraded":"2024-09-15T21:27:16.007276+0000","last_fullsized":"2024-09-15T21:27:16.007276+0000","mapping_epoch":45,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658904,"num_objects":406,"num_object_clones":0,"num_object_copies":812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18215","reported_seq":18303,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:15.042301+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:15.042301+0000","last_peered":"2024-09-15T21:27:15.042301+0000","last_clean":"2024-09-15T21:27:15.042301+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:15.042301+0000","last_undegraded":"2024-09-15T21:27:15.042301+0000","last_fullsized":"2024-09-15T21:27:15.042301+0000","mapping_epoch":36,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454080,"num_objects":355,"num_object_clones":0,"num_object_copies":710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18215,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18355","reported_seq":18492,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:15.064768+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:15.064768+0000","last_peered":"2024-09-15T21:27:15.064768+0000","last_clean":"2024-09-15T21:27:15.064768+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:15.064768+0000","last_undegraded":"2024-09-15T21:27:15.064768+0000","last_fullsized":"2024-09-15T21:27:15.064768+0000","mapping_epoch":35,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1507328,"num_objects":368,"num_object_clones":0,"num_object_copies":736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16213,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'17998","reported_seq":18830,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.116686+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:15.116686+0000","last_peered":"2024-09-15T21:27:15.116686+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:15.116686+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1568768,"num_objects":383,"num_object_clones":0,"num_object_copies":766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15423,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18523","reported_seq":19279,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.151447+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:15.151447+0000","last_peered":"2024-09-15T21:27:15.151447+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:15.151447+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560576,"num_objects":381,"num_object_clones":0,"num_object_copies":762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16159,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18040","reported_seq":18178,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:15.124956+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:15.124956+0000","last_peered":"2024-09-15T21:27:15.124956+0000","last_clean":"2024-09-15T21:27:15.124956+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:15.124956+0000","last_undegraded":"2024-09-15T21:27:15.124956+0000","last_fullsized":"2024-09-15T21:27:15.124956+0000","mapping_epoch":35,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1490944,"num_objects":364,"num_object_clones":0,"num_object_copies":728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15825,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18004","reported_seq":24836,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007642+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:16.007642+0000","last_peered":"2024-09-15T21:27:16.007642+0000","last_clean":"2024-09-15T21:27:16.007642+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:16.007642+0000","last_undegraded":"2024-09-15T21:27:16.007642+0000","last_fullsized":"2024-09-15T21:27:16.007642+0000","mapping_epoch":49,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1519616,"num_objects":371,"num_object_clones":0,"num_object_copies":742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17699","reported_seq":18492,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.147787+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:15.147787+0000","last_peered":"2024-09-15T21:27:15.147787+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:15.147787+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7600","ondisk_log_start":"46'7600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1372160,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15225,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'17884","reported_seq":24554,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:15.151718+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:15.151718+0000","last_peered":"2024-09-15T21:27:15.151718+0000","last_clean":"2024-09-15T21:27:15.151718+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:15.151718+0000","last_undegraded":"2024-09-15T21:27:15.151718+0000","last_fullsized":"2024-09-15T21:27:15.151718+0000","mapping_epoch":53,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1642496,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16435,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18163","reported_seq":18584,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.113260+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:15.113260+0000","last_peered":"2024-09-15T21:27:15.113260+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:15.113260+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1576960,"num_objects":385,"num_object_clones":0,"num_object_copies":770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18327","reported_seq":19080,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.933184+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:14.933184+0000","last_peered":"2024-09-15T21:27:14.933184+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:14.933184+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1568768,"num_objects":383,"num_object_clones":0,"num_object_copies":766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15634,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18092","reported_seq":25015,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.052145+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:16.052145+0000","last_peered":"2024-09-15T21:27:16.052145+0000","last_clean":"2024-09-15T21:27:16.052145+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:16.052145+0000","last_undegraded":"2024-09-15T21:27:16.052145+0000","last_fullsized":"2024-09-15T21:27:16.052145+0000","mapping_epoch":51,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18019","reported_seq":18687,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:15.136917+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:15.136917+0000","last_peered":"2024-09-15T21:27:15.136917+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:15.136917+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470464,"num_objects":359,"num_object_clones":0,"num_object_copies":718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15666,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2743}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18272","reported_seq":25375,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.028841+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:16.028841+0000","last_peered":"2024-09-15T21:27:16.028841+0000","last_clean":"2024-09-15T21:27:16.028841+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:16.028841+0000","last_undegraded":"2024-09-15T21:27:16.028841+0000","last_fullsized":"2024-09-15T21:27:16.028841+0000","mapping_epoch":47,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417833,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15932,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":24531585,"num_objects":5991,"num_object_clones":0,"num_object_copies":11982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261788,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19296,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":214777856,"data_stored":214721006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953594,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432660,"kb_used_data":121924,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93939180,"statfs":{"total":96636764160,"available":96193720320,"internally_reserved":0,"allocated":124850176,"data_stored":124538179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149820,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":71952,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":73678848,"data_stored":73370923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":428994225},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,1,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469176,"kb_used_data":18212,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93902664,"statfs":{"total":96636764160,"available":96156327936,"internally_reserved":0,"allocated":18649088,"data_stored":18298822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72998912,"data_stored":72995433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17719296,"data_stored":17673476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124059648,"data_stored":124052097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:20.094 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:20.983 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:20.983 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:21.042 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-15T21:27:20.592413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17420948,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263526,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19386,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395044,"kb_used_data":200496,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281720476,"statfs":{"total":289910292480,"available":288481767424,"internally_reserved":0,"allocated":205307904,"data_stored":204337716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,14,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24743936,"num_objects":-6041,"num_object_clones":0,"num_object_copies":-12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.053271"},"pg_stats":[{"pgid":"2.f","version":"54'18240","reported_seq":20024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.113351+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:20.113351+0000","last_peered":"2024-09-15T21:27:20.113351+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:20.113351+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18240,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18403","reported_seq":18518,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.173177+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:20.173177+0000","last_peered":"2024-09-15T21:27:20.173177+0000","last_clean":"2024-09-15T21:27:20.173177+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:20.173177+0000","last_undegraded":"2024-09-15T21:27:20.173177+0000","last_fullsized":"2024-09-15T21:27:20.173177+0000","mapping_epoch":36,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17129,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18464","reported_seq":25398,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007276+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:16.007276+0000","last_peered":"2024-09-15T21:27:16.007276+0000","last_clean":"2024-09-15T21:27:16.007276+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:16.007276+0000","last_undegraded":"2024-09-15T21:27:16.007276+0000","last_fullsized":"2024-09-15T21:27:16.007276+0000","mapping_epoch":45,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658904,"num_objects":406,"num_object_clones":0,"num_object_copies":812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18361","reported_seq":18449,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.144020+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:20.144020+0000","last_peered":"2024-09-15T21:27:20.144020+0000","last_clean":"2024-09-15T21:27:20.144020+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:20.144020+0000","last_undegraded":"2024-09-15T21:27:20.144020+0000","last_fullsized":"2024-09-15T21:27:20.144020+0000","mapping_epoch":36,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856064,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18361,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18506","reported_seq":18643,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.122055+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:20.122055+0000","last_peered":"2024-09-15T21:27:20.122055+0000","last_clean":"2024-09-15T21:27:20.122055+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:20.122055+0000","last_undegraded":"2024-09-15T21:27:20.122055+0000","last_fullsized":"2024-09-15T21:27:20.122055+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888832,"num_objects":217,"num_object_clones":0,"num_object_copies":434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18153","reported_seq":18985,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.147225+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:20.147225+0000","last_peered":"2024-09-15T21:27:20.147225+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:20.147225+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933888,"num_objects":228,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18667","reported_seq":19423,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.132275+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:20.132275+0000","last_peered":"2024-09-15T21:27:20.132275+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:20.132275+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18173","reported_seq":18311,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.051208+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:20.051208+0000","last_peered":"2024-09-15T21:27:20.051208+0000","last_clean":"2024-09-15T21:27:20.051208+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:20.051208+0000","last_undegraded":"2024-09-15T21:27:20.051208+0000","last_fullsized":"2024-09-15T21:27:20.051208+0000","mapping_epoch":35,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":946176,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15958,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18004","reported_seq":24836,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007642+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:16.007642+0000","last_peered":"2024-09-15T21:27:16.007642+0000","last_clean":"2024-09-15T21:27:16.007642+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:16.007642+0000","last_undegraded":"2024-09-15T21:27:16.007642+0000","last_fullsized":"2024-09-15T21:27:16.007642+0000","mapping_epoch":49,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1519616,"num_objects":371,"num_object_clones":0,"num_object_copies":742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17819","reported_seq":18612,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.043589+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:20.043589+0000","last_peered":"2024-09-15T21:27:20.043589+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:20.043589+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880640,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18025","reported_seq":24695,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.187234+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:20.187234+0000","last_peered":"2024-09-15T21:27:20.187234+0000","last_clean":"2024-09-15T21:27:20.187234+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:20.187234+0000","last_undegraded":"2024-09-15T21:27:20.187234+0000","last_fullsized":"2024-09-15T21:27:20.187234+0000","mapping_epoch":53,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1064960,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16576,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18318","reported_seq":18738,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.193853+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:20.193853+0000","last_peered":"2024-09-15T21:27:20.193853+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:20.193853+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16717,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18472","reported_seq":19222,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.114334+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:20.114334+0000","last_peered":"2024-09-15T21:27:20.114334+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:20.114334+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":974848,"num_objects":238,"num_object_clones":0,"num_object_copies":476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15779,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18092","reported_seq":25015,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.052145+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:16.052145+0000","last_peered":"2024-09-15T21:27:16.052145+0000","last_clean":"2024-09-15T21:27:16.052145+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:16.052145+0000","last_undegraded":"2024-09-15T21:27:16.052145+0000","last_fullsized":"2024-09-15T21:27:16.052145+0000","mapping_epoch":51,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18141","reported_seq":18806,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.161196+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:20.161196+0000","last_peered":"2024-09-15T21:27:20.161196+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:20.161196+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18272","reported_seq":25375,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.028841+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:16.028841+0000","last_peered":"2024-09-15T21:27:16.028841+0000","last_clean":"2024-09-15T21:27:16.028841+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:16.028841+0000","last_undegraded":"2024-09-15T21:27:16.028841+0000","last_fullsized":"2024-09-15T21:27:16.028841+0000","mapping_epoch":47,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417833,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15932,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17420929,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263524,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19385,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202706944,"data_stored":202650094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432144,"kb_used_data":117248,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93939696,"statfs":{"total":96636764160,"available":96194248704,"internally_reserved":0,"allocated":120061952,"data_stored":119749955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":322432685},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493724,"kb_used_data":65036,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93878116,"statfs":{"total":96636764160,"available":96131190784,"internally_reserved":0,"allocated":66596864,"data_stored":66288939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":438955697},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,14,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469176,"kb_used_data":18212,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93902664,"statfs":{"total":96636764160,"available":96156327936,"internally_reserved":0,"allocated":18649088,"data_stored":18298822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65957888,"data_stored":65954409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17719296,"data_stored":17673476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119029760,"data_stored":119022209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:21.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:21.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18240", 'reported_seq': 20024, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.113351+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:20.113351+0000', 'last_peered': '2024-09-15T21:27:20.113351+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:20.113351+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1114112, 'num_objects': 272, 'num_object_clones': 0, 'num_object_copies': 544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18240, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 844, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2251}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:21.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18153", 'reported_seq': 18985, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.147225+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:20.147225+0000', 'last_peered': '2024-09-15T21:27:20.147225+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:20.147225+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 933888, 'num_objects': 228, 'num_object_clones': 0, 'num_object_copies': 456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15578, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:21.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18667", 'reported_seq': 19423, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.132275+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:20.132275+0000', 'last_peered': '2024-09-15T21:27:20.132275+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:20.132275+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 970752, 'num_objects': 237, 'num_object_clones': 0, 'num_object_copies': 474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16303, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:21.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'17819", 'reported_seq': 18612, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.043589+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:20.043589+0000', 'last_peered': '2024-09-15T21:27:20.043589+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:20.043589+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7800", 'ondisk_log_start': "46'7800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 880640, 'num_objects': 215, 'num_object_clones': 0, 'num_object_copies': 430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15345, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2800}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:21.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18318", 'reported_seq': 18738, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.193853+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:20.193853+0000', 'last_peered': '2024-09-15T21:27:20.193853+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:20.193853+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 942080, 'num_objects': 230, 'num_object_clones': 0, 'num_object_copies': 460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16717, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2976}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:21.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:21.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18472", 'reported_seq': 19222, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.114334+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:20.114334+0000', 'last_peered': '2024-09-15T21:27:20.114334+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:20.114334+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8400", 'ondisk_log_start': "46'8400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 974848, 'num_objects': 238, 'num_object_clones': 0, 'num_object_copies': 476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15779, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:21.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18141", 'reported_seq': 18806, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:20.161196+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:20.161196+0000', 'last_peered': '2024-09-15T21:27:20.161196+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:20.161196+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8100", 'ondisk_log_start': "46'8100", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 970752, 'num_objects': 237, 'num_object_clones': 0, 'num_object_copies': 474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15788, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:21.047 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:21.861 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21180/8514 objects degraded (248.767%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":654,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":4257,"data_bytes":17420948,"bytes_used":1428525056,"bytes_avail":288481767424,"bytes_total":289910292480,"degraded_objects":21180,"degraded_total":8514,"degraded_ratio":2.4876673713883015,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":501},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:21.879 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:21.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:22.194 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:22.194 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:22.210 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-15T21:27:20.592413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17420948,"num_objects":4257,"num_object_clones":0,"num_object_copies":8514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4257,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263526,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19386,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395044,"kb_used_data":200496,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281720476,"statfs":{"total":289910292480,"available":288481767424,"internally_reserved":0,"allocated":205307904,"data_stored":204337716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,14,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24743936,"num_objects":-6041,"num_object_clones":0,"num_object_copies":-12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.053271"},"pg_stats":[{"pgid":"2.f","version":"54'18240","reported_seq":20024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.113351+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:20.113351+0000","last_peered":"2024-09-15T21:27:20.113351+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:20.113351+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18240,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18403","reported_seq":18518,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.173177+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:20.173177+0000","last_peered":"2024-09-15T21:27:20.173177+0000","last_clean":"2024-09-15T21:27:20.173177+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:20.173177+0000","last_undegraded":"2024-09-15T21:27:20.173177+0000","last_fullsized":"2024-09-15T21:27:20.173177+0000","mapping_epoch":36,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17129,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18464","reported_seq":25398,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007276+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:16.007276+0000","last_peered":"2024-09-15T21:27:16.007276+0000","last_clean":"2024-09-15T21:27:16.007276+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:16.007276+0000","last_undegraded":"2024-09-15T21:27:16.007276+0000","last_fullsized":"2024-09-15T21:27:16.007276+0000","mapping_epoch":45,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1658904,"num_objects":406,"num_object_clones":0,"num_object_copies":812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18361","reported_seq":18449,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.144020+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:20.144020+0000","last_peered":"2024-09-15T21:27:20.144020+0000","last_clean":"2024-09-15T21:27:20.144020+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:20.144020+0000","last_undegraded":"2024-09-15T21:27:20.144020+0000","last_fullsized":"2024-09-15T21:27:20.144020+0000","mapping_epoch":36,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856064,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18361,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18506","reported_seq":18643,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.122055+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:20.122055+0000","last_peered":"2024-09-15T21:27:20.122055+0000","last_clean":"2024-09-15T21:27:20.122055+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:20.122055+0000","last_undegraded":"2024-09-15T21:27:20.122055+0000","last_fullsized":"2024-09-15T21:27:20.122055+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888832,"num_objects":217,"num_object_clones":0,"num_object_copies":434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18153","reported_seq":18985,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.147225+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:20.147225+0000","last_peered":"2024-09-15T21:27:20.147225+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:20.147225+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933888,"num_objects":228,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18667","reported_seq":19423,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.132275+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:20.132275+0000","last_peered":"2024-09-15T21:27:20.132275+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:20.132275+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18173","reported_seq":18311,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.051208+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:20.051208+0000","last_peered":"2024-09-15T21:27:20.051208+0000","last_clean":"2024-09-15T21:27:20.051208+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:20.051208+0000","last_undegraded":"2024-09-15T21:27:20.051208+0000","last_fullsized":"2024-09-15T21:27:20.051208+0000","mapping_epoch":35,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":946176,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15958,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18004","reported_seq":24836,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.007642+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:16.007642+0000","last_peered":"2024-09-15T21:27:16.007642+0000","last_clean":"2024-09-15T21:27:16.007642+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:16.007642+0000","last_undegraded":"2024-09-15T21:27:16.007642+0000","last_fullsized":"2024-09-15T21:27:16.007642+0000","mapping_epoch":49,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1519616,"num_objects":371,"num_object_clones":0,"num_object_copies":742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16590,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17819","reported_seq":18612,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.043589+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:20.043589+0000","last_peered":"2024-09-15T21:27:20.043589+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:20.043589+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880640,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18025","reported_seq":24695,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.187234+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:20.187234+0000","last_peered":"2024-09-15T21:27:20.187234+0000","last_clean":"2024-09-15T21:27:20.187234+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:20.187234+0000","last_undegraded":"2024-09-15T21:27:20.187234+0000","last_fullsized":"2024-09-15T21:27:20.187234+0000","mapping_epoch":53,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1064960,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16576,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18318","reported_seq":18738,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.193853+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:20.193853+0000","last_peered":"2024-09-15T21:27:20.193853+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:20.193853+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16717,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18472","reported_seq":19222,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.114334+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:20.114334+0000","last_peered":"2024-09-15T21:27:20.114334+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:20.114334+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":974848,"num_objects":238,"num_object_clones":0,"num_object_copies":476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15779,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18092","reported_seq":25015,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.052145+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:16.052145+0000","last_peered":"2024-09-15T21:27:16.052145+0000","last_clean":"2024-09-15T21:27:16.052145+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:16.052145+0000","last_undegraded":"2024-09-15T21:27:16.052145+0000","last_fullsized":"2024-09-15T21:27:16.052145+0000","mapping_epoch":51,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18141","reported_seq":18806,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.161196+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:20.161196+0000","last_peered":"2024-09-15T21:27:20.161196+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:20.161196+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731060+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731060+0000","last_peered":"2024-09-15T21:26:58.731060+0000","last_clean":"2024-09-15T21:26:58.731060+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731060+0000","last_undegraded":"2024-09-15T21:26:58.731060+0000","last_fullsized":"2024-09-15T21:26:58.731060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18272","reported_seq":25375,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:16.028841+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:16.028841+0000","last_peered":"2024-09-15T21:27:16.028841+0000","last_clean":"2024-09-15T21:27:16.028841+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:16.028841+0000","last_undegraded":"2024-09-15T21:27:16.028841+0000","last_fullsized":"2024-09-15T21:27:16.028841+0000","mapping_epoch":47,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1417833,"num_objects":347,"num_object_clones":0,"num_object_copies":694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15932,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17420929,"num_objects":4255,"num_object_clones":0,"num_object_copies":8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263524,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19385,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202706944,"data_stored":202650094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432144,"kb_used_data":117248,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93939696,"statfs":{"total":96636764160,"available":96194248704,"internally_reserved":0,"allocated":120061952,"data_stored":119749955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":322432685},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493724,"kb_used_data":65036,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93878116,"statfs":{"total":96636764160,"available":96131190784,"internally_reserved":0,"allocated":66596864,"data_stored":66288939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":438955697},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,14,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626539,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469176,"kb_used_data":18212,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93902664,"statfs":{"total":96636764160,"available":96156327936,"internally_reserved":0,"allocated":18649088,"data_stored":18298822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65957888,"data_stored":65954409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17719296,"data_stored":17673476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119029760,"data_stored":119022209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:25.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:26.001 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:26.001 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:26.041 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-15T21:27:24.593256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14828180,"num_objects":3624,"num_object_clones":0,"num_object_copies":7248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264159,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19386,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391864,"kb_used_data":193160,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281723656,"statfs":{"total":289910292480,"available":288485023744,"internally_reserved":0,"allocated":197795840,"data_stored":196825652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,6,19,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18526208,"num_objects":-4523,"num_object_clones":0,"num_object_copies":-9046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4523,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.053178"},"pg_stats":[{"pgid":"2.f","version":"54'18240","reported_seq":20024,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.113351+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:20.113351+0000","last_peered":"2024-09-15T21:27:20.113351+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:20.113351+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18240,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18403","reported_seq":18518,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.173177+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:20.173177+0000","last_peered":"2024-09-15T21:27:20.173177+0000","last_clean":"2024-09-15T21:27:20.173177+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:20.173177+0000","last_undegraded":"2024-09-15T21:27:20.173177+0000","last_fullsized":"2024-09-15T21:27:20.173177+0000","mapping_epoch":36,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17129,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18640","reported_seq":25574,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:21.116586+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:21.116586+0000","last_peered":"2024-09-15T21:27:21.116586+0000","last_clean":"2024-09-15T21:27:21.116586+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:21.116586+0000","last_undegraded":"2024-09-15T21:27:21.116586+0000","last_fullsized":"2024-09-15T21:27:21.116586+0000","mapping_epoch":45,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":938008,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17363,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18361","reported_seq":18449,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.144020+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:20.144020+0000","last_peered":"2024-09-15T21:27:20.144020+0000","last_clean":"2024-09-15T21:27:20.144020+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:20.144020+0000","last_undegraded":"2024-09-15T21:27:20.144020+0000","last_fullsized":"2024-09-15T21:27:20.144020+0000","mapping_epoch":36,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856064,"num_objects":209,"num_object_clones":0,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18361,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18506","reported_seq":18643,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.122055+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:20.122055+0000","last_peered":"2024-09-15T21:27:20.122055+0000","last_clean":"2024-09-15T21:27:20.122055+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:20.122055+0000","last_undegraded":"2024-09-15T21:27:20.122055+0000","last_fullsized":"2024-09-15T21:27:20.122055+0000","mapping_epoch":35,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888832,"num_objects":217,"num_object_clones":0,"num_object_copies":434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18153","reported_seq":18985,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.147225+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:20.147225+0000","last_peered":"2024-09-15T21:27:20.147225+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:20.147225+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933888,"num_objects":228,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15578,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18667","reported_seq":19423,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.132275+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:20.132275+0000","last_peered":"2024-09-15T21:27:20.132275+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:20.132275+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18173","reported_seq":18311,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.051208+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:20.051208+0000","last_peered":"2024-09-15T21:27:20.051208+0000","last_clean":"2024-09-15T21:27:20.051208+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:20.051208+0000","last_undegraded":"2024-09-15T21:27:20.051208+0000","last_fullsized":"2024-09-15T21:27:20.051208+0000","mapping_epoch":35,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":946176,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15958,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18172","reported_seq":25004,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:21.168684+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:21.168684+0000","last_peered":"2024-09-15T21:27:21.168684+0000","last_clean":"2024-09-15T21:27:21.168684+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:21.168684+0000","last_undegraded":"2024-09-15T21:27:21.168684+0000","last_fullsized":"2024-09-15T21:27:21.168684+0000","mapping_epoch":49,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":831488,"num_objects":203,"num_object_clones":0,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17819","reported_seq":18612,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.043589+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:20.043589+0000","last_peered":"2024-09-15T21:27:20.043589+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:20.043589+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7800","ondisk_log_start":"46'7800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":880640,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15345,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2800}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18025","reported_seq":24695,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:20.187234+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:20.187234+0000","last_peered":"2024-09-15T21:27:20.187234+0000","last_clean":"2024-09-15T21:27:20.187234+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:20.187234+0000","last_undegraded":"2024-09-15T21:27:20.187234+0000","last_fullsized":"2024-09-15T21:27:20.187234+0000","mapping_epoch":53,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1064960,"num_objects":260,"num_object_clones":0,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16576,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18318","reported_seq":18738,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.193853+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:20.193853+0000","last_peered":"2024-09-15T21:27:20.193853+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:20.193853+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16717,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2976}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18472","reported_seq":19222,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.114334+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:20.114334+0000","last_peered":"2024-09-15T21:27:20.114334+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:20.114334+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":974848,"num_objects":238,"num_object_clones":0,"num_object_copies":476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15779,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18235","reported_seq":25158,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:21.062964+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:21.062964+0000","last_peered":"2024-09-15T21:27:21.062964+0000","last_clean":"2024-09-15T21:27:21.062964+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:21.062964+0000","last_undegraded":"2024-09-15T21:27:21.062964+0000","last_fullsized":"2024-09-15T21:27:21.062964+0000","mapping_epoch":51,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15820,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18141","reported_seq":18806,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:20.161196+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:20.161196+0000","last_peered":"2024-09-15T21:27:20.161196+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:20.161196+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":970752,"num_objects":237,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15788,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2865}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18418","reported_seq":25520,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:21.192026+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:21.192026+0000","last_peered":"2024-09-15T21:27:21.192026+0000","last_clean":"2024-09-15T21:27:21.192026+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:21.192026+0000","last_undegraded":"2024-09-15T21:27:21.192026+0000","last_fullsized":"2024-09-15T21:27:21.192026+0000","mapping_epoch":47,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":819817,"num_objects":201,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16078,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":14828161,"num_objects":3622,"num_object_clones":0,"num_object_copies":7244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264157,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19385,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195174400,"data_stored":195117550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432144,"kb_used_data":117248,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93939696,"statfs":{"total":96636764160,"available":96194248704,"internally_reserved":0,"allocated":120061952,"data_stored":119749955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":322432685},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493724,"kb_used_data":65036,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93878116,"statfs":{"total":96636764160,"available":96131190784,"internally_reserved":0,"allocated":66596864,"data_stored":66288939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":438955697},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,14,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626540,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465996,"kb_used_data":10876,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93905844,"statfs":{"total":96636764160,"available":96159584256,"internally_reserved":0,"allocated":11137024,"data_stored":10786758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65957888,"data_stored":65954409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10186752,"data_stored":10140932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119029760,"data_stored":119022209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:26.043 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:26.897 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:26.897 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:26.964 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-15T21:27:26.593664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5730964,"num_objects":1403,"num_object_clones":0,"num_object_copies":2806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266380,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19512,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1357544,"kb_used_data":175476,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281757976,"statfs":{"total":289910292480,"available":288520167424,"internally_reserved":0,"allocated":179687424,"data_stored":178717236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,7,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27623424,"num_objects":-6744,"num_object_clones":0,"num_object_copies":-13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6744,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.053163"},"pg_stats":[{"pgid":"2.f","version":"54'18399","reported_seq":20435,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.244143+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:25.244143+0000","last_peered":"2024-09-15T21:27:25.244143+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:25.244143+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462848,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18399,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":970,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2284}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18539","reported_seq":18654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.203244+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:25.203244+0000","last_peered":"2024-09-15T21:27:25.203244+0000","last_clean":"2024-09-15T21:27:25.203244+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:25.203244+0000","last_undegraded":"2024-09-15T21:27:25.203244+0000","last_fullsized":"2024-09-15T21:27:25.203244+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413696,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18783","reported_seq":25717,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.115994+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:26.115994+0000","last_peered":"2024-09-15T21:27:26.115994+0000","last_clean":"2024-09-15T21:27:26.115994+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:26.115994+0000","last_undegraded":"2024-09-15T21:27:26.115994+0000","last_fullsized":"2024-09-15T21:27:26.115994+0000","mapping_epoch":45,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352280,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17506,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18494","reported_seq":18582,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.231766+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:25.231766+0000","last_peered":"2024-09-15T21:27:25.231766+0000","last_clean":"2024-09-15T21:27:25.231766+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:25.231766+0000","last_undegraded":"2024-09-15T21:27:25.231766+0000","last_fullsized":"2024-09-15T21:27:25.231766+0000","mapping_epoch":36,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18494,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18635","reported_seq":18772,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.212659+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:25.212659+0000","last_peered":"2024-09-15T21:27:25.212659+0000","last_clean":"2024-09-15T21:27:25.212659+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:25.212659+0000","last_undegraded":"2024-09-15T21:27:25.212659+0000","last_fullsized":"2024-09-15T21:27:25.212659+0000","mapping_epoch":35,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16493,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18269","reported_seq":19101,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.129486+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:25.129486+0000","last_peered":"2024-09-15T21:27:25.129486+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:25.129486+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15694,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18823","reported_seq":19579,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.184248+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:25.184248+0000","last_peered":"2024-09-15T21:27:25.184248+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:25.184248+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331776,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16459,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18309","reported_seq":18447,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.204371+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:25.204371+0000","last_peered":"2024-09-15T21:27:25.204371+0000","last_clean":"2024-09-15T21:27:25.204371+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:25.204371+0000","last_undegraded":"2024-09-15T21:27:25.204371+0000","last_fullsized":"2024-09-15T21:27:25.204371+0000","mapping_epoch":35,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18311","reported_seq":25143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.116469+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:26.116469+0000","last_peered":"2024-09-15T21:27:26.116469+0000","last_clean":"2024-09-15T21:27:26.116469+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:26.116469+0000","last_undegraded":"2024-09-15T21:27:26.116469+0000","last_fullsized":"2024-09-15T21:27:26.116469+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16897,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17946","reported_seq":18739,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.186658+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:25.186658+0000","last_peered":"2024-09-15T21:27:25.186658+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:25.186658+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18182","reported_seq":24852,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.237333+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:25.237333+0000","last_peered":"2024-09-15T21:27:25.237333+0000","last_clean":"2024-09-15T21:27:25.237333+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:25.237333+0000","last_undegraded":"2024-09-15T21:27:25.237333+0000","last_fullsized":"2024-09-15T21:27:25.237333+0000","mapping_epoch":53,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16733,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18463","reported_seq":18884,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.221187+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:25.221187+0000","last_peered":"2024-09-15T21:27:25.221187+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:25.221187+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348160,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18611","reported_seq":19361,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.243303+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:25.243303+0000","last_peered":"2024-09-15T21:27:25.243303+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:25.243303+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":405504,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15918,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18360","reported_seq":25283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.251126+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:26.251126+0000","last_peered":"2024-09-15T21:27:26.251126+0000","last_clean":"2024-09-15T21:27:26.251126+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:26.251126+0000","last_undegraded":"2024-09-15T21:27:26.251126+0000","last_fullsized":"2024-09-15T21:27:26.251126+0000","mapping_epoch":51,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18281","reported_seq":18949,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.139349+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:25.139349+0000","last_peered":"2024-09-15T21:27:25.139349+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:25.139349+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15928,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18559","reported_seq":25662,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.215290+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:26.215290+0000","last_peered":"2024-09-15T21:27:26.215290+0000","last_clean":"2024-09-15T21:27:26.215290+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:26.215290+0000","last_undegraded":"2024-09-15T21:27:26.215290+0000","last_fullsized":"2024-09-15T21:27:26.215290+0000","mapping_epoch":47,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242281,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16219,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":5730945,"num_objects":1401,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266378,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19511,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176885760,"data_stored":176828910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415120,"kb_used_data":112704,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93956720,"statfs":{"total":96636764160,"available":96211681280,"internally_reserved":0,"allocated":115408896,"data_stored":115096899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149822,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491404,"kb_used_data":58552,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93880436,"statfs":{"total":96636764160,"available":96133566464,"internally_reserved":0,"allocated":59957248,"data_stored":59649323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":443215537},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,6,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451020,"kb_used_data":4220,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93920820,"statfs":{"total":96636764160,"available":96174919680,"internally_reserved":0,"allocated":4321280,"data_stored":3971014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":59158528,"data_stored":59155049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3256320,"data_stored":3210500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114470912,"data_stored":114463361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:26.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:26.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18399", 'reported_seq': 20435, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.244143+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:25.244143+0000', 'last_peered': '2024-09-15T21:27:25.244143+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:25.244143+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 462848, 'num_objects': 113, 'num_object_clones': 0, 'num_object_copies': 226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2397, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18399, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 970, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2284}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:26.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18269", 'reported_seq': 19101, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.129486+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:25.129486+0000', 'last_peered': '2024-09-15T21:27:25.129486+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:25.129486+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 458752, 'num_objects': 112, 'num_object_clones': 0, 'num_object_copies': 224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15694, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:26.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18823", 'reported_seq': 19579, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.184248+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:25.184248+0000', 'last_peered': '2024-09-15T21:27:25.184248+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:25.184248+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8800", 'ondisk_log_start': "46'8800", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 331776, 'num_objects': 81, 'num_object_clones': 0, 'num_object_copies': 162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 81, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16459, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'17946", 'reported_seq': 18739, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.186658+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:25.186658+0000', 'last_peered': '2024-09-15T21:27:25.186658+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:25.186658+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'7900", 'ondisk_log_start': "46'7900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 360448, 'num_objects': 88, 'num_object_clones': 0, 'num_object_copies': 176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 88, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15472, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2927}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18463", 'reported_seq': 18884, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.221187+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:25.221187+0000', 'last_peered': '2024-09-15T21:27:25.221187+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:25.221187+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8400", 'ondisk_log_start': "46'8400", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 348160, 'num_objects': 85, 'num_object_clones': 0, 'num_object_copies': 170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 85, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16862, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3121}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18611", 'reported_seq': 19361, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.243303+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:25.243303+0000', 'last_peered': '2024-09-15T21:27:25.243303+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:25.243303+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8600", 'ondisk_log_start': "46'8600", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 405504, 'num_objects': 99, 'num_object_clones': 0, 'num_object_copies': 198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 99, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15918, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3031}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:26.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18281", 'reported_seq': 18949, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:25.139349+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:25.139349+0000', 'last_peered': '2024-09-15T21:27:25.139349+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:25.139349+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8200", 'ondisk_log_start': "46'8200", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 397312, 'num_objects': 97, 'num_object_clones': 0, 'num_object_copies': 194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 97, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15928, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:26.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:27.474 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:27.474 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21054/2806 objects degraded (750.321%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":660,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":1403,"data_bytes":5730964,"bytes_used":1390125056,"bytes_avail":288520167424,"bytes_total":289910292480,"degraded_objects":21054,"degraded_total":2806,"degraded_ratio":7.5032074126871002,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":345,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:27.525 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:27.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:28.356 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:28.358 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:28.391 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-15T21:27:26.593664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5730964,"num_objects":1403,"num_object_clones":0,"num_object_copies":2806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266380,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19512,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1357544,"kb_used_data":175476,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281757976,"statfs":{"total":289910292480,"available":288520167424,"internally_reserved":0,"allocated":179687424,"data_stored":178717236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,7,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27623424,"num_objects":-6744,"num_object_clones":0,"num_object_copies":-13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6744,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.053163"},"pg_stats":[{"pgid":"2.f","version":"54'18399","reported_seq":20435,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.244143+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:25.244143+0000","last_peered":"2024-09-15T21:27:25.244143+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:25.244143+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462848,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18399,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":970,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2284}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18539","reported_seq":18654,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.203244+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:25.203244+0000","last_peered":"2024-09-15T21:27:25.203244+0000","last_clean":"2024-09-15T21:27:25.203244+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:25.203244+0000","last_undegraded":"2024-09-15T21:27:25.203244+0000","last_fullsized":"2024-09-15T21:27:25.203244+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413696,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18783","reported_seq":25717,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.115994+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:26.115994+0000","last_peered":"2024-09-15T21:27:26.115994+0000","last_clean":"2024-09-15T21:27:26.115994+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:26.115994+0000","last_undegraded":"2024-09-15T21:27:26.115994+0000","last_fullsized":"2024-09-15T21:27:26.115994+0000","mapping_epoch":45,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352280,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17506,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18494","reported_seq":18582,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.231766+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:25.231766+0000","last_peered":"2024-09-15T21:27:25.231766+0000","last_clean":"2024-09-15T21:27:25.231766+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:25.231766+0000","last_undegraded":"2024-09-15T21:27:25.231766+0000","last_fullsized":"2024-09-15T21:27:25.231766+0000","mapping_epoch":36,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18494,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18635","reported_seq":18772,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.212659+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:25.212659+0000","last_peered":"2024-09-15T21:27:25.212659+0000","last_clean":"2024-09-15T21:27:25.212659+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:25.212659+0000","last_undegraded":"2024-09-15T21:27:25.212659+0000","last_fullsized":"2024-09-15T21:27:25.212659+0000","mapping_epoch":35,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16493,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18269","reported_seq":19101,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.129486+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:25.129486+0000","last_peered":"2024-09-15T21:27:25.129486+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:25.129486+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15694,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18823","reported_seq":19579,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.184248+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:25.184248+0000","last_peered":"2024-09-15T21:27:25.184248+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:25.184248+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331776,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16459,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18309","reported_seq":18447,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.204371+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:25.204371+0000","last_peered":"2024-09-15T21:27:25.204371+0000","last_clean":"2024-09-15T21:27:25.204371+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:25.204371+0000","last_undegraded":"2024-09-15T21:27:25.204371+0000","last_fullsized":"2024-09-15T21:27:25.204371+0000","mapping_epoch":35,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389120,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18311","reported_seq":25143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.116469+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:26.116469+0000","last_peered":"2024-09-15T21:27:26.116469+0000","last_clean":"2024-09-15T21:27:26.116469+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:26.116469+0000","last_undegraded":"2024-09-15T21:27:26.116469+0000","last_fullsized":"2024-09-15T21:27:26.116469+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16897,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'17946","reported_seq":18739,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.186658+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:25.186658+0000","last_peered":"2024-09-15T21:27:25.186658+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:25.186658+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'7900","ondisk_log_start":"46'7900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360448,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18182","reported_seq":24852,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:25.237333+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:25.237333+0000","last_peered":"2024-09-15T21:27:25.237333+0000","last_clean":"2024-09-15T21:27:25.237333+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:25.237333+0000","last_undegraded":"2024-09-15T21:27:25.237333+0000","last_fullsized":"2024-09-15T21:27:25.237333+0000","mapping_epoch":53,"log_start":"46'8100","ondisk_log_start":"46'8100","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421888,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16733,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18463","reported_seq":18884,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.221187+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:25.221187+0000","last_peered":"2024-09-15T21:27:25.221187+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:25.221187+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348160,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3121}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18611","reported_seq":19361,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.243303+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:25.243303+0000","last_peered":"2024-09-15T21:27:25.243303+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:25.243303+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":405504,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15918,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18360","reported_seq":25283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.251126+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:26.251126+0000","last_peered":"2024-09-15T21:27:26.251126+0000","last_clean":"2024-09-15T21:27:26.251126+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:26.251126+0000","last_undegraded":"2024-09-15T21:27:26.251126+0000","last_fullsized":"2024-09-15T21:27:26.251126+0000","mapping_epoch":51,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18281","reported_seq":18949,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:25.139349+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:25.139349+0000","last_peered":"2024-09-15T21:27:25.139349+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:25.139349+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397312,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15928,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18559","reported_seq":25662,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.215290+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:26.215290+0000","last_peered":"2024-09-15T21:27:26.215290+0000","last_clean":"2024-09-15T21:27:26.215290+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:26.215290+0000","last_undegraded":"2024-09-15T21:27:26.215290+0000","last_fullsized":"2024-09-15T21:27:26.215290+0000","mapping_epoch":47,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242281,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16219,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":5730945,"num_objects":1401,"num_object_clones":0,"num_object_copies":2802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266378,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19511,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176885760,"data_stored":176828910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415120,"kb_used_data":112704,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93956720,"statfs":{"total":96636764160,"available":96211681280,"internally_reserved":0,"allocated":115408896,"data_stored":115096899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149822,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491404,"kb_used_data":58552,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93880436,"statfs":{"total":96636764160,"available":96133566464,"internally_reserved":0,"allocated":59957248,"data_stored":59649323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":443215537},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,6,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451020,"kb_used_data":4220,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93920820,"statfs":{"total":96636764160,"available":96174919680,"internally_reserved":0,"allocated":4321280,"data_stored":3971014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":59158528,"data_stored":59155049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3256320,"data_stored":3210500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114470912,"data_stored":114463361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:28.641 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Removed 148140 objects 2024-09-15T21:27:28.641 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi114.stdout:Clean up completed and total clean up time :335.074 2024-09-15T21:27:31.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:31.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:31.760 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:31.781 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-15T21:27:30.598949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1069716,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267518,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19679,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353504,"kb_used_data":167280,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281762016,"statfs":{"total":289910292480,"available":288524304384,"internally_reserved":0,"allocated":171294720,"data_stored":170328604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,2,1,9,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23461888,"num_objects":-5728,"num_object_clones":0,"num_object_copies":-11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.057638"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":20882,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:30.257465+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:30.257465+0000","last_peered":"2024-09-15T21:27:30.257465+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:30.257465+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18783","reported_seq":25717,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.115994+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:26.115994+0000","last_peered":"2024-09-15T21:27:26.115994+0000","last_clean":"2024-09-15T21:27:26.115994+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:26.115994+0000","last_undegraded":"2024-09-15T21:27:26.115994+0000","last_fullsized":"2024-09-15T21:27:26.115994+0000","mapping_epoch":45,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352280,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17506,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18311","reported_seq":25143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.116469+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:26.116469+0000","last_peered":"2024-09-15T21:27:26.116469+0000","last_clean":"2024-09-15T21:27:26.116469+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:26.116469+0000","last_undegraded":"2024-09-15T21:27:26.116469+0000","last_fullsized":"2024-09-15T21:27:26.116469+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16897,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18360","reported_seq":25283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.251126+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:26.251126+0000","last_peered":"2024-09-15T21:27:26.251126+0000","last_clean":"2024-09-15T21:27:26.251126+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:26.251126+0000","last_undegraded":"2024-09-15T21:27:26.251126+0000","last_fullsized":"2024-09-15T21:27:26.251126+0000","mapping_epoch":51,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18559","reported_seq":25662,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.215290+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:26.215290+0000","last_peered":"2024-09-15T21:27:26.215290+0000","last_clean":"2024-09-15T21:27:26.215290+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:26.215290+0000","last_undegraded":"2024-09-15T21:27:26.215290+0000","last_fullsized":"2024-09-15T21:27:26.215290+0000","mapping_epoch":47,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242281,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16219,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1069697,"num_objects":263,"num_object_clones":0,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267516,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19678,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169320448,"data_stored":169267670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451020,"kb_used_data":4220,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93920820,"statfs":{"total":96636764160,"available":96174919680,"internally_reserved":0,"allocated":4321280,"data_stored":3971014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3256320,"data_stored":3210500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:31.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:32.099 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:32.099 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:32.118 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-15T21:27:30.598949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1069716,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267518,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19679,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353504,"kb_used_data":167280,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281762016,"statfs":{"total":289910292480,"available":288524304384,"internally_reserved":0,"allocated":171294720,"data_stored":170328604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,2,1,9,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23461888,"num_objects":-5728,"num_object_clones":0,"num_object_copies":-11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.057638"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":20882,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:30.257465+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:30.257465+0000","last_peered":"2024-09-15T21:27:30.257465+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:30.257465+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18783","reported_seq":25717,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.115994+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:26.115994+0000","last_peered":"2024-09-15T21:27:26.115994+0000","last_clean":"2024-09-15T21:27:26.115994+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:26.115994+0000","last_undegraded":"2024-09-15T21:27:26.115994+0000","last_fullsized":"2024-09-15T21:27:26.115994+0000","mapping_epoch":45,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352280,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17506,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18311","reported_seq":25143,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.116469+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:26.116469+0000","last_peered":"2024-09-15T21:27:26.116469+0000","last_clean":"2024-09-15T21:27:26.116469+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:26.116469+0000","last_undegraded":"2024-09-15T21:27:26.116469+0000","last_fullsized":"2024-09-15T21:27:26.116469+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16897,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18360","reported_seq":25283,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.251126+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:26.251126+0000","last_peered":"2024-09-15T21:27:26.251126+0000","last_clean":"2024-09-15T21:27:26.251126+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:26.251126+0000","last_undegraded":"2024-09-15T21:27:26.251126+0000","last_fullsized":"2024-09-15T21:27:26.251126+0000","mapping_epoch":51,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18559","reported_seq":25662,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:26.215290+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:26.215290+0000","last_peered":"2024-09-15T21:27:26.215290+0000","last_clean":"2024-09-15T21:27:26.215290+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:26.215290+0000","last_undegraded":"2024-09-15T21:27:26.215290+0000","last_fullsized":"2024-09-15T21:27:26.215290+0000","mapping_epoch":47,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242281,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16219,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1069697,"num_objects":263,"num_object_clones":0,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267516,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19678,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169320448,"data_stored":169267670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626541,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451020,"kb_used_data":4220,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93920820,"statfs":{"total":96636764160,"available":96174919680,"internally_reserved":0,"allocated":4321280,"data_stored":3971014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3256320,"data_stored":3210500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 20882, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:30.257465+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:30.257465+0000', 'last_peered': '2024-09-15T21:27:30.257465+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:30.257465+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1137, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2230}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:32.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18827, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596507+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.596507+0000', 'last_peered': '2024-09-15T21:27:28.596507+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.596507+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18969, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.586170+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.586170+0000', 'last_peered': '2024-09-15T21:27:28.586170+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.586170+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:32.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:32.121 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:32.495 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:32.495 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20887/530 objects degraded (3940.943%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":665,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":265,"data_bytes":1069716,"bytes_used":1385988096,"bytes_avail":288524304384,"bytes_total":289910292480,"degraded_objects":20887,"degraded_total":530,"degraded_ratio":39.409433962264153,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":382,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:32.515 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:32.515 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:32.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:32.841 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:32.859 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-15T21:27:32.599630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19679,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17420312,"num_objects":-4254,"num_object_clones":0,"num_object_copies":-8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007216"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":20882,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:30.257465+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:30.257465+0000","last_peered":"2024-09-15T21:27:30.257465+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:30.257465+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19678,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626542,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:35.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:36.197 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:36.215 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-15T21:27:34.600105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19679,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14827544,"num_objects":-3621,"num_object_clones":0,"num_object_copies":-7242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3621,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007262"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":20882,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:30.257465+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:30.257465+0000","last_peered":"2024-09-15T21:27:30.257465+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:30.257465+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19678,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626542,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:36.217 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:36.568 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:36.569 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:36.588 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-15T21:27:34.600105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19679,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14827544,"num_objects":-3621,"num_object_clones":0,"num_object_copies":-7242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3621,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007262"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":20882,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:30.257465+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:30.257465+0000","last_peered":"2024-09-15T21:27:30.257465+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:30.257465+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1137,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2230}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19678,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953597,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149823,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626542,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:36.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:36.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 20882, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:30.257465+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:30.257465+0000', 'last_peered': '2024-09-15T21:27:30.257465+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:30.257465+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1137, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2230}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:36.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18827, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596507+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.596507+0000', 'last_peered': '2024-09-15T21:27:28.596507+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.596507+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18969, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.586170+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.586170+0000', 'last_peered': '2024-09-15T21:27:28.586170+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.586170+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:36.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:36.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:36.993 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20513/6 objects degraded (341883.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":670,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1390690304,"bytes_avail":288519602176,"bytes_total":289910292480,"degraded_objects":20513,"degraded_total":6,"degraded_ratio":3418.8333333333339,"recovering_objects_per_sec":55,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":301},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:37.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:37.016 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:37.348 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:37.349 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:37.370 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-15T21:27:36.600716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20053,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358096,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757424,"statfs":{"total":289910292480,"available":288519602176,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14827544,"num_objects":-3621,"num_object_clones":0,"num_object_copies":-7242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3621,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007459"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":21630,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:35.251661+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:35.251661+0000","last_peered":"2024-09-15T21:27:35.251661+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:35.251661+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1511,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18969,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.586170+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.586170+0000","last_peered":"2024-09-15T21:27:28.586170+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.586170+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20052,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953598,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149824,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455612,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916228,"statfs":{"total":96636764160,"available":96170217472,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:40.372 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:40.712 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:40.712 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:40.730 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-15T21:27:40.601964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20492,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358096,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757424,"statfs":{"total":289910292480,"available":288519602176,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5730328,"num_objects":-1400,"num_object_clones":0,"num_object_copies":-2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":22508,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:40.260659+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:40.260659+0000","last_peered":"2024-09-15T21:27:40.260659+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:40.260659+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20491,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455612,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916228,"statfs":{"total":96636764160,"available":96170217472,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:40.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:41.393 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:41.394 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:41.412 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-15T21:27:40.601964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20492,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358096,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757424,"statfs":{"total":289910292480,"available":288519602176,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5730328,"num_objects":-1400,"num_object_clones":0,"num_object_copies":-2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":22508,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:40.260659+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:40.260659+0000","last_peered":"2024-09-15T21:27:40.260659+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:40.260659+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20491,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455612,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916228,"statfs":{"total":96636764160,"available":96170217472,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:41.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:41.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 22508, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:40.260659+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:40.260659+0000', 'last_peered': '2024-09-15T21:27:40.260659+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:40.260659+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1417, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1950, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1417}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18827, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596507+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.596507+0000', 'last_peered': '2024-09-15T21:27:28.596507+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.596507+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18970, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.632895+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.632895+0000', 'last_peered': '2024-09-15T21:27:28.632895+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.632895+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:41.414 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:41.415 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:41.810 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:41.810 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20074/6 objects degraded (334566.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1390690304,"bytes_avail":288519602176,"bytes_total":289910292480,"degraded_objects":20074,"degraded_total":6,"degraded_ratio":3345.6666666666661,"recovering_objects_per_sec":81,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":980,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":116},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:41.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:41.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:42.148 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:42.167 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-15T21:27:40.601964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20492,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358096,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757424,"statfs":{"total":289910292480,"available":288519602176,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5730328,"num_objects":-1400,"num_object_clones":0,"num_object_copies":-2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":22508,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:40.260659+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:40.260659+0000","last_peered":"2024-09-15T21:27:40.260659+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:40.260659+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25804,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.636101+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.636101+0000","last_peered":"2024-09-15T21:27:28.636101+0000","last_clean":"2024-09-15T21:27:28.636101+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.636101+0000","last_undegraded":"2024-09-15T21:27:28.636101+0000","last_fullsized":"2024-09-15T21:27:28.636101+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20491,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455612,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916228,"statfs":{"total":96636764160,"available":96170217472,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:45.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:45.506 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:45.507 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:45.525 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-15T21:27:44.603096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20492,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1345616,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281769904,"statfs":{"total":289910292480,"available":288532381696,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-813,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":22508,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:40.260659+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:40.260659+0000","last_peered":"2024-09-15T21:27:40.260659+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:40.260659+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20491,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:45.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:46.067 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:46.068 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:46.087 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-15T21:27:44.603096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20492,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1345616,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281769904,"statfs":{"total":289910292480,"available":288532381696,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-813,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":22508,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:40.260659+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:40.260659+0000","last_peered":"2024-09-15T21:27:40.260659+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:40.260659+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1417,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25207,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.590726+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.590726+0000","last_peered":"2024-09-15T21:27:28.590726+0000","last_clean":"2024-09-15T21:27:28.590726+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.590726+0000","last_undegraded":"2024-09-15T21:27:28.590726+0000","last_fullsized":"2024-09-15T21:27:28.590726+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20491,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953599,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149825,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626544,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443132,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928708,"statfs":{"total":96636764160,"available":96182996992,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:46.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:46.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 22508, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:40.260659+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:40.260659+0000', 'last_peered': '2024-09-15T21:27:40.260659+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:40.260659+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1417, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1950, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1417}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:46.089 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18827, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596507+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.596507+0000', 'last_peered': '2024-09-15T21:27:28.596507+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.596507+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:46.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18970, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.632895+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.632895+0000', 'last_peered': '2024-09-15T21:27:28.632895+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.632895+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:46.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:46.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:46.091 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:46.091 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:46.497 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:46.497 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20074/6 objects degraded (334566.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1377910784,"bytes_avail":288532381696,"bytes_total":289910292480,"degraded_objects":20074,"degraded_total":6,"degraded_ratio":3345.6666666666661,"recovering_objects_per_sec":67,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:46.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:46.518 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:46.843 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:46.862 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-15T21:27:46.603910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20993,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349776,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765744,"statfs":{"total":289910292480,"available":288528121856,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1314,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1314,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003804"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":23510,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:45.264103+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:45.264103+0000","last_peered":"2024-09-15T21:27:45.264103+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:45.264103+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20992,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149826,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447292,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924548,"statfs":{"total":96636764160,"available":96178737152,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:49.865 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:50.194 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:50.216 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-15T21:27:48.604342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20993,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349776,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765744,"statfs":{"total":289910292480,"available":288528121856,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":940,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003626"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":23510,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:45.264103+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:45.264103+0000","last_peered":"2024-09-15T21:27:45.264103+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:45.264103+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20992,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149826,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447292,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924548,"statfs":{"total":96636764160,"available":96178737152,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:50.217 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:50.538 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:50.538 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:50.559 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-15T21:27:48.604342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20993,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349776,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765744,"statfs":{"total":289910292480,"available":288528121856,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":940,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003626"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":23510,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:45.264103+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:45.264103+0000","last_peered":"2024-09-15T21:27:45.264103+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:45.264103+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18827,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596507+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.596507+0000","last_peered":"2024-09-15T21:27:28.596507+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.596507+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":119,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748371+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748371+0000","last_peered":"2024-09-15T21:26:58.748371+0000","last_clean":"2024-09-15T21:26:58.748371+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748371+0000","last_undegraded":"2024-09-15T21:26:58.748371+0000","last_fullsized":"2024-09-15T21:26:58.748371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20992,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953600,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149826,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447292,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924548,"statfs":{"total":96636764160,"available":96178737152,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 23510, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:45.264103+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:45.264103+0000', 'last_peered': '2024-09-15T21:27:45.264103+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:45.264103+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 916, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2451, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -916}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18827, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596507+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.596507+0000', 'last_peered': '2024-09-15T21:27:28.596507+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.596507+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18970, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.632895+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.632895+0000', 'last_peered': '2024-09-15T21:27:28.632895+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.632895+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:50.562 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:50.562 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:50.947 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:50.947 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18979/6 objects degraded (316316.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":683,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1382170624,"bytes_avail":288528121856,"bytes_total":289910292480,"degraded_objects":18979,"degraded_total":6,"degraded_ratio":3163.1666666666661,"recovering_objects_per_sec":127,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1534,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:50.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:50.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:51.280 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:51.300 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-15T21:27:50.604967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21587,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349776,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765744,"statfs":{"total":289910292480,"available":288528121856,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1534,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003698"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":24698,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:50.267870+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:50.267870+0000","last_peered":"2024-09-15T21:27:50.267870+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:50.267870+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3045,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18828,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.617713+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.617713+0000","last_peered":"2024-09-15T21:27:28.617713+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.617713+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748468+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748468+0000","last_peered":"2024-09-15T21:26:58.748468+0000","last_clean":"2024-09-15T21:26:58.748468+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748468+0000","last_undegraded":"2024-09-15T21:26:58.748468+0000","last_fullsized":"2024-09-15T21:26:58.748468+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21586,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626545,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447292,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924548,"statfs":{"total":96636764160,"available":96178737152,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:54.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:54.709 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:54.709 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:54.729 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-15T21:27:54.606073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21587,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":594,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002163"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":24698,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:50.267870+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:50.267870+0000","last_peered":"2024-09-15T21:27:50.267870+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:50.267870+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3045,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18828,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.617713+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.617713+0000","last_peered":"2024-09-15T21:27:28.617713+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.617713+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748468+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748468+0000","last_peered":"2024-09-15T21:26:58.748468+0000","last_clean":"2024-09-15T21:26:58.748468+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748468+0000","last_undegraded":"2024-09-15T21:26:58.748468+0000","last_fullsized":"2024-09-15T21:26:58.748468+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21586,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:54.730 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:55.106 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:55.106 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:55.128 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-15T21:27:54.606073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21587,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":594,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002163"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":24698,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:50.267870+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:50.267870+0000","last_peered":"2024-09-15T21:27:50.267870+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:50.267870+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3045,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18828,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.617713+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.617713+0000","last_peered":"2024-09-15T21:27:28.617713+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.617713+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748468+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748468+0000","last_peered":"2024-09-15T21:26:58.748468+0000","last_clean":"2024-09-15T21:26:58.748468+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748468+0000","last_undegraded":"2024-09-15T21:26:58.748468+0000","last_fullsized":"2024-09-15T21:26:58.748468+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21586,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "54'18512", 'reported_seq': 24698, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:50.267870+0000', 'last_change': '2024-09-15T21:26:57.547414+0000', 'last_active': '2024-09-15T21:27:50.267870+0000', 'last_peered': '2024-09-15T21:27:50.267870+0000', 'last_clean': '2024-09-15T21:18:41.352229+0000', 'last_became_active': '2024-09-15T21:18:43.846845+0000', 'last_became_peered': '2024-09-15T21:18:43.846845+0000', 'last_unstale': '2024-09-15T21:27:50.267870+0000', 'last_undegraded': '2024-09-15T21:18:43.502067+0000', 'last_fullsized': '2024-09-15T21:18:43.370091+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'objects_scrubbed': 0, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 322, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18512, 'num_write_kb': 37024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3045, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -322}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19213, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.623704+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:28.623704+0000', 'last_peered': '2024-09-15T21:27:28.623704+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:28.623704+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:55.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19660, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.596864+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:28.596864+0000', 'last_peered': '2024-09-15T21:27:28.596864+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:28.596864+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18828, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.617713+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:28.617713+0000', 'last_peered': '2024-09-15T21:27:28.617713+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:28.617713+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 18970, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.632895+0000', 'last_change': '2024-09-15T21:18:43.855446+0000', 'last_active': '2024-09-15T21:27:28.632895+0000', 'last_peered': '2024-09-15T21:27:28.632895+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:28.632895+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3208, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 144, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19463, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.589606+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:28.589606+0000', 'last_peered': '2024-09-15T21:27:28.589606+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:28.589606+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:55.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19046, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.637104+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:28.637104+0000', 'last_peered': '2024-09-15T21:27:28.637104+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:28.637104+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:55.131 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:55.505 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18979/6 objects degraded (316316.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":688,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1386430464,"bytes_avail":288523862016,"bytes_total":289910292480,"degraded_objects":18979,"degraded_total":6,"degraded_ratio":3163.1666666666661,"recovering_objects_per_sec":74,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":594,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:55.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:55.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:55.862 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:55.862 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:55.881 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-15T21:27:54.606073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21587,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353936,"kb_used_data":163552,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761584,"statfs":{"total":289910292480,"available":288523862016,"internally_reserved":0,"allocated":167477248,"data_stored":166515204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":594,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002163"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":24698,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:50.267870+0000","last_change":"2024-09-15T21:26:57.547414+0000","last_active":"2024-09-15T21:27:50.267870+0000","last_peered":"2024-09-15T21:27:50.267870+0000","last_clean":"2024-09-15T21:18:41.352229+0000","last_became_active":"2024-09-15T21:18:43.846845+0000","last_became_peered":"2024-09-15T21:18:43.846845+0000","last_unstale":"2024-09-15T21:27:50.267870+0000","last_undegraded":"2024-09-15T21:18:43.502067+0000","last_fullsized":"2024-09-15T21:18:43.370091+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3045,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18755,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.616755+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:28.616755+0000","last_peered":"2024-09-15T21:27:28.616755+0000","last_clean":"2024-09-15T21:27:28.616755+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:28.616755+0000","last_undegraded":"2024-09-15T21:27:28.616755+0000","last_fullsized":"2024-09-15T21:27:28.616755+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25805,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.649035+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:28.649035+0000","last_peered":"2024-09-15T21:27:28.649035+0000","last_clean":"2024-09-15T21:27:28.649035+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:28.649035+0000","last_undegraded":"2024-09-15T21:27:28.649035+0000","last_fullsized":"2024-09-15T21:27:28.649035+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18658,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.506897+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:28.506897+0000","last_peered":"2024-09-15T21:27:28.506897+0000","last_clean":"2024-09-15T21:27:28.506897+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:28.506897+0000","last_undegraded":"2024-09-15T21:27:28.506897+0000","last_fullsized":"2024-09-15T21:27:28.506897+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18860,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.611398+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:28.611398+0000","last_peered":"2024-09-15T21:27:28.611398+0000","last_clean":"2024-09-15T21:27:28.611398+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:28.611398+0000","last_undegraded":"2024-09-15T21:27:28.611398+0000","last_fullsized":"2024-09-15T21:27:28.611398+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19213,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.623704+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:28.623704+0000","last_peered":"2024-09-15T21:27:28.623704+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:28.623704+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19660,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.596864+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:28.596864+0000","last_peered":"2024-09-15T21:27:28.596864+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:28.596864+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18542,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.580334+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:28.580334+0000","last_peered":"2024-09-15T21:27:28.580334+0000","last_clean":"2024-09-15T21:27:28.580334+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:28.580334+0000","last_undegraded":"2024-09-15T21:27:28.580334+0000","last_fullsized":"2024-09-15T21:27:28.580334+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.729686+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:26:58.729686+0000","last_peered":"2024-09-15T21:26:58.729686+0000","last_clean":"2024-09-15T21:26:58.729686+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:26:58.729686+0000","last_undegraded":"2024-09-15T21:26:58.729686+0000","last_fullsized":"2024-09-15T21:26:58.729686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.602549+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:28.602549+0000","last_peered":"2024-09-15T21:27:28.602549+0000","last_clean":"2024-09-15T21:27:28.602549+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:28.602549+0000","last_undegraded":"2024-09-15T21:27:28.602549+0000","last_fullsized":"2024-09-15T21:27:28.602549+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730217+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:26:58.730217+0000","last_peered":"2024-09-15T21:26:58.730217+0000","last_clean":"2024-09-15T21:26:58.730217+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:26:58.730217+0000","last_undegraded":"2024-09-15T21:26:58.730217+0000","last_fullsized":"2024-09-15T21:26:58.730217+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18828,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.617713+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:28.617713+0000","last_peered":"2024-09-15T21:27:28.617713+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:28.617713+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.748468+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:26:58.748468+0000","last_peered":"2024-09-15T21:26:58.748468+0000","last_clean":"2024-09-15T21:26:58.748468+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:26:58.748468+0000","last_undegraded":"2024-09-15T21:26:58.748468+0000","last_fullsized":"2024-09-15T21:26:58.748468+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24955,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.637836+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:28.637836+0000","last_peered":"2024-09-15T21:27:28.637836+0000","last_clean":"2024-09-15T21:27:28.637836+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:28.637836+0000","last_undegraded":"2024-09-15T21:27:28.637836+0000","last_fullsized":"2024-09-15T21:27:28.637836+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":98,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.730782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:26:58.730782+0000","last_peered":"2024-09-15T21:26:58.730782+0000","last_clean":"2024-09-15T21:26:58.730782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:26:58.730782+0000","last_undegraded":"2024-09-15T21:26:58.730782+0000","last_fullsized":"2024-09-15T21:26:58.730782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":18970,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.632895+0000","last_change":"2024-09-15T21:18:43.855446+0000","last_active":"2024-09-15T21:27:28.632895+0000","last_peered":"2024-09-15T21:27:28.632895+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:28.632895+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19463,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.589606+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:28.589606+0000","last_peered":"2024-09-15T21:27:28.589606+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:28.589606+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731241+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:26:58.731241+0000","last_peered":"2024-09-15T21:26:58.731241+0000","last_clean":"2024-09-15T21:26:58.731241+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:26:58.731241+0000","last_undegraded":"2024-09-15T21:26:58.731241+0000","last_fullsized":"2024-09-15T21:26:58.731241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.535723+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:28.535723+0000","last_peered":"2024-09-15T21:27:28.535723+0000","last_clean":"2024-09-15T21:27:28.535723+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:28.535723+0000","last_undegraded":"2024-09-15T21:27:28.535723+0000","last_fullsized":"2024-09-15T21:27:28.535723+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.732449+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:26:58.732449+0000","last_peered":"2024-09-15T21:26:58.732449+0000","last_clean":"2024-09-15T21:26:58.732449+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:26:58.732449+0000","last_undegraded":"2024-09-15T21:26:58.732449+0000","last_fullsized":"2024-09-15T21:26:58.732449+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19046,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.637104+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:28.637104+0000","last_peered":"2024-09-15T21:27:28.637104+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:28.637104+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731121+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:26:58.731121+0000","last_peered":"2024-09-15T21:26:58.731121+0000","last_clean":"2024-09-15T21:26:58.731121+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:26:58.731121+0000","last_undegraded":"2024-09-15T21:26:58.731121+0000","last_fullsized":"2024-09-15T21:26:58.731121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.731156+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:26:58.731156+0000","last_peered":"2024-09-15T21:26:58.731156+0000","last_clean":"2024-09-15T21:26:58.731156+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:26:58.731156+0000","last_undegraded":"2024-09-15T21:26:58.731156+0000","last_fullsized":"2024-09-15T21:26:58.731156+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25721,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.629862+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:28.629862+0000","last_peered":"2024-09-15T21:27:28.629862+0000","last_clean":"2024-09-15T21:27:28.629862+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:28.629862+0000","last_undegraded":"2024-09-15T21:27:28.629862+0000","last_fullsized":"2024-09-15T21:27:28.629862+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21586,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166113280,"data_stored":166064574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411644,"kb_used_data":109228,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960196,"statfs":{"total":96636764160,"available":96215240704,"internally_reserved":0,"allocated":111849472,"data_stored":111541547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490840,"kb_used_data":53832,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93881000,"statfs":{"total":96636764160,"available":96134144000,"internally_reserved":0,"allocated":55123968,"data_stored":54816043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451452,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920388,"statfs":{"total":96636764160,"available":96174477312,"internally_reserved":0,"allocated":503808,"data_stored":157614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:58.883 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:59.263 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:59.264 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:59.285 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-15T21:27:58.607246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22272,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002279"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":19701,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:55.265370+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:27:55.265370+0000","last_peered":"2024-09-15T21:27:55.265370+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:55.265370+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2845,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":507,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2845}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19050,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.803820+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.803820+0000","last_peered":"2024-09-15T21:27:53.803820+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.803820+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22271,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149828,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:59.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:59.663 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:27:59.663 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:27:59.681 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-15T21:27:58.607246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22272,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002279"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":19701,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:55.265370+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:27:55.265370+0000","last_peered":"2024-09-15T21:27:55.265370+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:27:55.265370+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2845,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":507,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2845}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19050,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.803820+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.803820+0000","last_peered":"2024-09-15T21:27:53.803820+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.803820+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22271,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953602,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149828,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:59.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:59.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19217, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799801+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:53.799801+0000', 'last_peered': '2024-09-15T21:27:53.799801+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:53.799801+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19664, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.800282+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:53.800282+0000', 'last_peered': '2024-09-15T21:27:53.800282+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:53.800282+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18832, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:53.799911+0000', 'last_peered': '2024-09-15T21:27:53.799911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:53.799911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 19701, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:55.265370+0000', 'last_change': '2024-09-15T21:27:52.327228+0000', 'last_active': '2024-09-15T21:27:55.265370+0000', 'last_peered': '2024-09-15T21:27:55.265370+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:27:55.265370+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2845, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 507, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2845}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19467, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.804029+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:53.804029+0000', 'last_peered': '2024-09-15T21:27:53.804029+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:53.804029+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:59.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19050, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.803820+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:53.803820+0000', 'last_peered': '2024-09-15T21:27:53.803820+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:53.803820+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:59.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:00.242 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:00.242 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18294/6 objects degraded (304900.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":693,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1390743552,"bytes_avail":288519548928,"bytes_total":289910292480,"degraded_objects":18294,"degraded_total":6,"degraded_ratio":3049,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:00.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:00.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:00.657 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:00.658 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:00.675 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-15T21:28:00.607656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23124,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002259"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":21405,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:00.260861+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:00.260861+0000","last_peered":"2024-09-15T21:28:00.260861+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:00.260861+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23123,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:03.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:04.013 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:04.032 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-15T21:28:02.608326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23124,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349828,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765692,"statfs":{"total":289910292480,"available":288528068608,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.815477"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":21405,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:00.260861+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:00.260861+0000","last_peered":"2024-09-15T21:28:00.260861+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:00.260861+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23123,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447300,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924540,"statfs":{"total":96636764160,"available":96178728960,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:04.033 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:04.349 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:04.350 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:04.368 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-15T21:28:02.608326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23124,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349828,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765692,"statfs":{"total":289910292480,"available":288528068608,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.815477"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":21405,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:00.260861+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:00.260861+0000","last_peered":"2024-09-15T21:28:00.260861+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:00.260861+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23123,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447300,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924540,"statfs":{"total":96636764160,"available":96178728960,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:04.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:04.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19217, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799801+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:53.799801+0000', 'last_peered': '2024-09-15T21:27:53.799801+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:53.799801+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:04.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19664, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.800282+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:53.800282+0000', 'last_peered': '2024-09-15T21:27:53.800282+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:53.800282+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18832, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:53.799911+0000', 'last_peered': '2024-09-15T21:27:53.799911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:53.799911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 21405, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:00.260861+0000', 'last_change': '2024-09-15T21:27:52.327228+0000', 'last_active': '2024-09-15T21:28:00.260861+0000', 'last_peered': '2024-09-15T21:28:00.260861+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:28:00.260861+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1993, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1359, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1993}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19467, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.804029+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:53.804029+0000', 'last_peered': '2024-09-15T21:27:53.804029+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:53.804029+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:04.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19051, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.805124+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:53.805124+0000', 'last_peered': '2024-09-15T21:27:53.805124+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:53.805124+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.371 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:04.749 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17442/6 objects degraded (290700.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":697,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1382223872,"bytes_avail":288528068608,"bytes_total":289910292480,"degraded_objects":17442,"degraded_total":6,"degraded_ratio":2907,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-15T21:26:20.524419+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:04.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:04.769 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:05.905 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:05.906 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:05.931 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-15T21:28:04.609235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23124,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349828,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765692,"statfs":{"total":289910292480,"available":288528068608,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.812287"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801068+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801068+0000","last_peered":"2024-09-15T21:27:53.801068+0000","last_clean":"2024-09-15T21:27:53.801068+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801068+0000","last_undegraded":"2024-09-15T21:27:53.801068+0000","last_fullsized":"2024-09-15T21:27:53.801068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24959,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801286+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801286+0000","last_peered":"2024-09-15T21:27:53.801286+0000","last_clean":"2024-09-15T21:27:53.801286+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801286+0000","last_undegraded":"2024-09-15T21:27:53.801286+0000","last_fullsized":"2024-09-15T21:27:53.801286+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":21405,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:00.260861+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:00.260861+0000","last_peered":"2024-09-15T21:28:00.260861+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:00.260861+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1359,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1993}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801887+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801887+0000","last_peered":"2024-09-15T21:27:53.801887+0000","last_clean":"2024-09-15T21:27:53.801887+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801887+0000","last_undegraded":"2024-09-15T21:27:53.801887+0000","last_fullsized":"2024-09-15T21:27:53.801887+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23123,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953603,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149829,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626548,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447300,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924540,"statfs":{"total":96636764160,"available":96178728960,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:08.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:09.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:28:09.315+0000 7f8e9b59e640 0 --2- 172.21.15.114:0/2154121809 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f8e9c12b400 0x7f8e9c1be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:28:09.558 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:09.559 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:09.583 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-15T21:28:08.610325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23546,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353988,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761532,"statfs":{"total":289910292480,"available":288523808768,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003838"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":22249,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:05.270002+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:05.270002+0000","last_peered":"2024-09-15T21:28:05.270002+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:05.270002+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1781,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23545,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149830,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451460,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920380,"statfs":{"total":96636764160,"available":96174469120,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:09.585 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:09.922 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:09.923 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:09.941 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-15T21:28:08.610325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23546,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353988,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761532,"statfs":{"total":289910292480,"available":288523808768,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003838"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":22249,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:05.270002+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:05.270002+0000","last_peered":"2024-09-15T21:28:05.270002+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:05.270002+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1781,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23545,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149830,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451460,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920380,"statfs":{"total":96636764160,"available":96174469120,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:09.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:09.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19217, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799801+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:53.799801+0000', 'last_peered': '2024-09-15T21:27:53.799801+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:53.799801+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:09.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19664, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.800282+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:53.800282+0000', 'last_peered': '2024-09-15T21:27:53.800282+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:53.800282+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18832, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:53.799911+0000', 'last_peered': '2024-09-15T21:27:53.799911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:53.799911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 22249, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:05.270002+0000', 'last_change': '2024-09-15T21:27:52.327228+0000', 'last_active': '2024-09-15T21:28:05.270002+0000', 'last_peered': '2024-09-15T21:28:05.270002+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:28:05.270002+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1571, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1781, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1571}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19467, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.804029+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:53.804029+0000', 'last_peered': '2024-09-15T21:27:53.804029+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:53.804029+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:09.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19051, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.805124+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:53.805124+0000', 'last_peered': '2024-09-15T21:27:53.805124+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:53.805124+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.945 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:10.321 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:10.322 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17020/6 objects degraded (283666.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1386483712,"bytes_avail":288523808768,"bytes_total":289910292480,"degraded_objects":17020,"degraded_total":6,"degraded_ratio":2836.6666666666661,"recovering_objects_per_sec":106,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1274,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:10.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:10.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:10.661 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:10.682 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-15T21:28:10.610888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23958,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1353988,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281761532,"statfs":{"total":289910292480,"available":288523808768,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003642"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":23073,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:10.268155+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:10.268155+0000","last_peered":"2024-09-15T21:28:10.268155+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:10.268155+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23957,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626549,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451460,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920380,"statfs":{"total":96636764160,"available":96174469120,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:13.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:14.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:28:14.019+0000 7f0a3a000640 0 --2- 172.21.15.114:0/2730107160 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f0a34073260 0x7f0a34073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:28:14.239 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:14.239 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:14.259 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-15T21:28:12.611286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23958,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003630"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":23073,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:10.268155+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:10.268155+0000","last_peered":"2024-09-15T21:28:10.268155+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:10.268155+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23957,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:14.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:14.586 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:14.587 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:14.606 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-15T21:28:12.611286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23958,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003630"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":23073,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:10.268155+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:10.268155+0000","last_peered":"2024-09-15T21:28:10.268155+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:10.268155+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23957,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:14.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:14.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19217, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799801+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:53.799801+0000', 'last_peered': '2024-09-15T21:27:53.799801+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:53.799801+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:14.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:14.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19664, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.800282+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:53.800282+0000', 'last_peered': '2024-09-15T21:27:53.800282+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:53.800282+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:14.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18832, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:53.799911+0000', 'last_peered': '2024-09-15T21:27:53.799911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:53.799911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 23073, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:10.268155+0000', 'last_change': '2024-09-15T21:27:52.327228+0000', 'last_active': '2024-09-15T21:28:10.268155+0000', 'last_peered': '2024-09-15T21:28:10.268155+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:28:10.268155+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1159, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2193, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19467, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.804029+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:53.804029+0000', 'last_peered': '2024-09-15T21:27:53.804029+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:53.804029+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:14.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19051, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.805124+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:53.805124+0000', 'last_peered': '2024-09-15T21:27:53.805124+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:53.805124+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:14.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:14.985 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:14.985 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16608/6 objects degraded (276800.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1390743552,"bytes_avail":288519548928,"bytes_total":289910292480,"degraded_objects":16608,"degraded_total":6,"degraded_ratio":2768,"recovering_objects_per_sec":69,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":834,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:15.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:15.005 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:15.332 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:15.333 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:15.351 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-15T21:28:14.611944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23958,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1358148,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281757372,"statfs":{"total":289910292480,"available":288519548928,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-834,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18546,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799646+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799646+0000","last_peered":"2024-09-15T21:27:53.799646+0000","last_clean":"2024-09-15T21:27:53.799646+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799646+0000","last_undegraded":"2024-09-15T21:27:53.799646+0000","last_fullsized":"2024-09-15T21:27:53.799646+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":23073,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:10.268155+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:10.268155+0000","last_peered":"2024-09-15T21:28:10.268155+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:10.268155+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23957,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953605,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455620,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916220,"statfs":{"total":96636764160,"available":96170209280,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:18.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:18.728 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:18.728 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:18.746 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-15T21:28:18.613147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24503,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1345668,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281769852,"statfs":{"total":289910292480,"available":288532328448,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18547,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799759+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799759+0000","last_peered":"2024-09-15T21:27:53.799759+0000","last_clean":"2024-09-15T21:27:53.799759+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799759+0000","last_undegraded":"2024-09-15T21:27:53.799759+0000","last_fullsized":"2024-09-15T21:27:53.799759+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":24163,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:15.274746+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:15.274746+0000","last_peered":"2024-09-15T21:28:15.274746+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:15.274746+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24502,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443140,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928700,"statfs":{"total":96636764160,"available":96182988800,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:18.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:19.058 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:19.058 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:19.077 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-15T21:28:18.613147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24503,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1345668,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281769852,"statfs":{"total":289910292480,"available":288532328448,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18547,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799759+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799759+0000","last_peered":"2024-09-15T21:27:53.799759+0000","last_clean":"2024-09-15T21:27:53.799759+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799759+0000","last_undegraded":"2024-09-15T21:27:53.799759+0000","last_fullsized":"2024-09-15T21:27:53.799759+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":24163,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:15.274746+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:15.274746+0000","last_peered":"2024-09-15T21:28:15.274746+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:15.274746+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24502,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443140,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928700,"statfs":{"total":96636764160,"available":96182988800,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19217, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799801+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:27:53.799801+0000', 'last_peered': '2024-09-15T21:27:53.799801+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:27:53.799801+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19664, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.800282+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:27:53.800282+0000', 'last_peered': '2024-09-15T21:27:53.800282+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:27:53.800282+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:19.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18832, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.799911+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:27:53.799911+0000', 'last_peered': '2024-09-15T21:27:53.799911+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:27:53.799911+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 24163, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:15.274746+0000', 'last_change': '2024-09-15T21:27:52.327228+0000', 'last_active': '2024-09-15T21:28:15.274746+0000', 'last_peered': '2024-09-15T21:28:15.274746+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:28:15.274746+0000', 'last_undegraded': '2024-09-15T21:18:43.448670+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 43, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 614, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2738, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -614}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19467, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.804029+0000', 'last_change': '2024-09-15T21:18:43.890093+0000', 'last_active': '2024-09-15T21:27:53.804029+0000', 'last_peered': '2024-09-15T21:27:53.804029+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:27:53.804029+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 303, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:19.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19051, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.805124+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:27:53.805124+0000', 'last_peered': '2024-09-15T21:27:53.805124+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:27:53.805124+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:19.079 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:19.460 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:19.460 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16063/6 objects degraded (267716.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1377964032,"bytes_avail":288532328448,"bytes_total":289910292480,"degraded_objects":16063,"degraded_total":6,"degraded_ratio":2677.1666666666661,"recovering_objects_per_sec":79,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":957,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:19.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:19.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:19.842 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:19.843 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:19.869 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-15T21:28:19.702785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24503,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1345668,"kb_used_data":163592,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281769852,"statfs":{"total":289910292480,"available":288532328448,"internally_reserved":0,"allocated":167518208,"data_stored":166531187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1210371082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.092459"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25353,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.880999+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:27:53.880999+0000","last_peered":"2024-09-15T21:27:53.880999+0000","last_clean":"2024-09-15T21:27:53.880999+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:27:53.880999+0000","last_undegraded":"2024-09-15T21:27:53.880999+0000","last_fullsized":"2024-09-15T21:27:53.880999+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18759,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801704+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:27:53.801704+0000","last_peered":"2024-09-15T21:27:53.801704+0000","last_clean":"2024-09-15T21:27:53.801704+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:27:53.801704+0000","last_undegraded":"2024-09-15T21:27:53.801704+0000","last_fullsized":"2024-09-15T21:27:53.801704+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25809,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:27:53.805458+0000","last_peered":"2024-09-15T21:27:53.805458+0000","last_clean":"2024-09-15T21:27:53.805458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:27:53.805458+0000","last_undegraded":"2024-09-15T21:27:53.805458+0000","last_fullsized":"2024-09-15T21:27:53.805458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18662,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801586+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:27:53.801586+0000","last_peered":"2024-09-15T21:27:53.801586+0000","last_clean":"2024-09-15T21:27:53.801586+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:27:53.801586+0000","last_undegraded":"2024-09-15T21:27:53.801586+0000","last_fullsized":"2024-09-15T21:27:53.801586+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18864,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.803423+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:27:53.803423+0000","last_peered":"2024-09-15T21:27:53.803423+0000","last_clean":"2024-09-15T21:27:53.803423+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:27:53.803423+0000","last_undegraded":"2024-09-15T21:27:53.803423+0000","last_fullsized":"2024-09-15T21:27:53.803423+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19217,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799801+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:27:53.799801+0000","last_peered":"2024-09-15T21:27:53.799801+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:27:53.799801+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19664,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.800282+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:27:53.800282+0000","last_peered":"2024-09-15T21:27:53.800282+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:27:53.800282+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18547,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799759+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:27:53.799759+0000","last_peered":"2024-09-15T21:27:53.799759+0000","last_clean":"2024-09-15T21:27:53.799759+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:27:53.799759+0000","last_undegraded":"2024-09-15T21:27:53.799759+0000","last_fullsized":"2024-09-15T21:27:53.799759+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.804723+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:27:53.804723+0000","last_peered":"2024-09-15T21:27:53.804723+0000","last_clean":"2024-09-15T21:27:53.804723+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:27:53.804723+0000","last_undegraded":"2024-09-15T21:27:53.804723+0000","last_fullsized":"2024-09-15T21:27:53.804723+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25212,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805156+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:27:53.805156+0000","last_peered":"2024-09-15T21:27:53.805156+0000","last_clean":"2024-09-15T21:27:53.805156+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:27:53.805156+0000","last_undegraded":"2024-09-15T21:27:53.805156+0000","last_fullsized":"2024-09-15T21:27:53.805156+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801168+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:27:53.801168+0000","last_peered":"2024-09-15T21:27:53.801168+0000","last_clean":"2024-09-15T21:27:53.801168+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:27:53.801168+0000","last_undegraded":"2024-09-15T21:27:53.801168+0000","last_fullsized":"2024-09-15T21:27:53.801168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18832,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.799911+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:27:53.799911+0000","last_peered":"2024-09-15T21:27:53.799911+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:27:53.799911+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":124,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.799682+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:27:53.799682+0000","last_peered":"2024-09-15T21:27:53.799682+0000","last_clean":"2024-09-15T21:27:53.799682+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:27:53.799682+0000","last_undegraded":"2024-09-15T21:27:53.799682+0000","last_fullsized":"2024-09-15T21:27:53.799682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24960,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801374+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:27:53.801374+0000","last_peered":"2024-09-15T21:27:53.801374+0000","last_clean":"2024-09-15T21:27:53.801374+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:27:53.801374+0000","last_undegraded":"2024-09-15T21:27:53.801374+0000","last_fullsized":"2024-09-15T21:27:53.801374+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":102,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805103+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:27:53.805103+0000","last_peered":"2024-09-15T21:27:53.805103+0000","last_clean":"2024-09-15T21:27:53.805103+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:27:53.805103+0000","last_undegraded":"2024-09-15T21:27:53.805103+0000","last_fullsized":"2024-09-15T21:27:53.805103+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":24163,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:15.274746+0000","last_change":"2024-09-15T21:27:52.327228+0000","last_active":"2024-09-15T21:28:15.274746+0000","last_peered":"2024-09-15T21:28:15.274746+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:15.274746+0000","last_undegraded":"2024-09-15T21:18:43.448670+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":43,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2738,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-614}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19467,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.804029+0000","last_change":"2024-09-15T21:18:43.890093+0000","last_active":"2024-09-15T21:27:53.804029+0000","last_peered":"2024-09-15T21:27:53.804029+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:27:53.804029+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805358+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:27:53.805358+0000","last_peered":"2024-09-15T21:27:53.805358+0000","last_clean":"2024-09-15T21:27:53.805358+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:27:53.805358+0000","last_undegraded":"2024-09-15T21:27:53.805358+0000","last_fullsized":"2024-09-15T21:27:53.805358+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25339,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805704+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:27:53.805704+0000","last_peered":"2024-09-15T21:27:53.805704+0000","last_clean":"2024-09-15T21:27:53.805704+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:27:53.805704+0000","last_undegraded":"2024-09-15T21:27:53.805704+0000","last_fullsized":"2024-09-15T21:27:53.805704+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.801927+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:27:53.801927+0000","last_peered":"2024-09-15T21:27:53.801927+0000","last_clean":"2024-09-15T21:27:53.801927+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:27:53.801927+0000","last_undegraded":"2024-09-15T21:27:53.801927+0000","last_fullsized":"2024-09-15T21:27:53.801927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19051,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.805124+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:27:53.805124+0000","last_peered":"2024-09-15T21:27:53.805124+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:27:53.805124+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805626+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:27:53.805626+0000","last_peered":"2024-09-15T21:27:53.805626+0000","last_clean":"2024-09-15T21:27:53.805626+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:27:53.805626+0000","last_undegraded":"2024-09-15T21:27:53.805626+0000","last_fullsized":"2024-09-15T21:27:53.805626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805107+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:27:53.805107+0000","last_peered":"2024-09-15T21:27:53.805107+0000","last_clean":"2024-09-15T21:27:53.805107+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:27:53.805107+0000","last_undegraded":"2024-09-15T21:27:53.805107+0000","last_fullsized":"2024-09-15T21:27:53.805107+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25725,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.805268+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:27:53.805268+0000","last_peered":"2024-09-15T21:27:53.805268+0000","last_clean":"2024-09-15T21:27:53.805268+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:27:53.805268+0000","last_undegraded":"2024-09-15T21:27:53.805268+0000","last_fullsized":"2024-09-15T21:27:53.805268+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16063,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24502,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166105088,"data_stored":166063340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953606,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149832,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490864,"kb_used_data":53848,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880976,"statfs":{"total":96636764160,"available":96134119424,"internally_reserved":0,"allocated":55140352,"data_stored":54821782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443140,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928700,"statfs":{"total":96636764160,"available":96182988800,"internally_reserved":0,"allocated":512000,"data_stored":162119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111394816,"data_stored":111391337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:22.872 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:23.273 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:23.296 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-15T21:28:22.614190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25209,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349844,"kb_used_data":163612,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765676,"statfs":{"total":289910292480,"available":288528052224,"internally_reserved":0,"allocated":167538688,"data_stored":166539114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002245"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25355,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.710690+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:19.710690+0000","last_peered":"2024-09-15T21:28:19.710690+0000","last_clean":"2024-09-15T21:28:19.710690+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:19.710690+0000","last_undegraded":"2024-09-15T21:28:19.710690+0000","last_fullsized":"2024-09-15T21:28:19.710690+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18761,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.705716+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:19.705716+0000","last_peered":"2024-09-15T21:28:19.705716+0000","last_clean":"2024-09-15T21:28:19.705716+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:19.705716+0000","last_undegraded":"2024-09-15T21:28:19.705716+0000","last_fullsized":"2024-09-15T21:28:19.705716+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18664,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.707205+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:19.707205+0000","last_peered":"2024-09-15T21:28:19.707205+0000","last_clean":"2024-09-15T21:28:19.707205+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:19.707205+0000","last_undegraded":"2024-09-15T21:28:19.707205+0000","last_fullsized":"2024-09-15T21:28:19.707205+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18866,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704465+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:19.704465+0000","last_peered":"2024-09-15T21:28:19.704465+0000","last_clean":"2024-09-15T21:28:19.704465+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:19.704465+0000","last_undegraded":"2024-09-15T21:28:19.704465+0000","last_fullsized":"2024-09-15T21:28:19.704465+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19219,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703821+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:19.703821+0000","last_peered":"2024-09-15T21:28:19.703821+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:19.703821+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19666,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703750+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:19.703750+0000","last_peered":"2024-09-15T21:28:19.703750+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:19.703750+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18549,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704347+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:19.704347+0000","last_peered":"2024-09-15T21:28:19.704347+0000","last_clean":"2024-09-15T21:28:19.704347+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:19.704347+0000","last_undegraded":"2024-09-15T21:28:19.704347+0000","last_fullsized":"2024-09-15T21:28:19.704347+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704186+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:19.704186+0000","last_peered":"2024-09-15T21:28:19.704186+0000","last_clean":"2024-09-15T21:28:19.704186+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:19.704186+0000","last_undegraded":"2024-09-15T21:28:19.704186+0000","last_fullsized":"2024-09-15T21:28:19.704186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18834,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.704601+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:19.704601+0000","last_peered":"2024-09-15T21:28:19.704601+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:19.704601+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":126,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.703696+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:19.703696+0000","last_peered":"2024-09-15T21:28:19.703696+0000","last_clean":"2024-09-15T21:28:19.703696+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:19.703696+0000","last_undegraded":"2024-09-15T21:28:19.703696+0000","last_fullsized":"2024-09-15T21:28:19.703696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24962,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704266+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:19.704266+0000","last_peered":"2024-09-15T21:28:19.704266+0000","last_clean":"2024-09-15T21:28:19.704266+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:19.704266+0000","last_undegraded":"2024-09-15T21:28:19.704266+0000","last_fullsized":"2024-09-15T21:28:19.704266+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25393,"reported_epoch":56,"state":"peering","last_fresh":"2024-09-15T21:28:19.670949+0000","last_change":"2024-09-15T21:28:19.712116+0000","last_active":"2024-09-15T21:28:19.712116+0000","last_peered":"2024-09-15T21:28:19.670949+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:19.670949+0000","last_undegraded":"2024-09-15T21:28:19.670949+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19654,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.275488+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:20.275488+0000","last_peered":"2024-09-15T21:28:20.275488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:20.275488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704820+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:19.704820+0000","last_peered":"2024-09-15T21:28:19.704820+0000","last_clean":"2024-09-15T21:28:19.704820+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:19.704820+0000","last_undegraded":"2024-09-15T21:28:19.704820+0000","last_fullsized":"2024-09-15T21:28:19.704820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19053,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.705383+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:19.705383+0000","last_peered":"2024-09-15T21:28:19.705383+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:19.705383+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25208,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490868,"kb_used_data":53856,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880972,"statfs":{"total":96636764160,"available":96134115328,"internally_reserved":0,"allocated":55148544,"data_stored":54824637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:23.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:23.662 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:23.663 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:23.686 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-15T21:28:22.614190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25209,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349844,"kb_used_data":163612,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765676,"statfs":{"total":289910292480,"available":288528052224,"internally_reserved":0,"allocated":167538688,"data_stored":166539114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002245"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25355,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.710690+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:19.710690+0000","last_peered":"2024-09-15T21:28:19.710690+0000","last_clean":"2024-09-15T21:28:19.710690+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:19.710690+0000","last_undegraded":"2024-09-15T21:28:19.710690+0000","last_fullsized":"2024-09-15T21:28:19.710690+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18761,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.705716+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:19.705716+0000","last_peered":"2024-09-15T21:28:19.705716+0000","last_clean":"2024-09-15T21:28:19.705716+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:19.705716+0000","last_undegraded":"2024-09-15T21:28:19.705716+0000","last_fullsized":"2024-09-15T21:28:19.705716+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18664,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.707205+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:19.707205+0000","last_peered":"2024-09-15T21:28:19.707205+0000","last_clean":"2024-09-15T21:28:19.707205+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:19.707205+0000","last_undegraded":"2024-09-15T21:28:19.707205+0000","last_fullsized":"2024-09-15T21:28:19.707205+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18866,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704465+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:19.704465+0000","last_peered":"2024-09-15T21:28:19.704465+0000","last_clean":"2024-09-15T21:28:19.704465+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:19.704465+0000","last_undegraded":"2024-09-15T21:28:19.704465+0000","last_fullsized":"2024-09-15T21:28:19.704465+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19219,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703821+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:19.703821+0000","last_peered":"2024-09-15T21:28:19.703821+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:19.703821+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19666,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703750+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:19.703750+0000","last_peered":"2024-09-15T21:28:19.703750+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:19.703750+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18549,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704347+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:19.704347+0000","last_peered":"2024-09-15T21:28:19.704347+0000","last_clean":"2024-09-15T21:28:19.704347+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:19.704347+0000","last_undegraded":"2024-09-15T21:28:19.704347+0000","last_fullsized":"2024-09-15T21:28:19.704347+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704186+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:19.704186+0000","last_peered":"2024-09-15T21:28:19.704186+0000","last_clean":"2024-09-15T21:28:19.704186+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:19.704186+0000","last_undegraded":"2024-09-15T21:28:19.704186+0000","last_fullsized":"2024-09-15T21:28:19.704186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18834,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.704601+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:19.704601+0000","last_peered":"2024-09-15T21:28:19.704601+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:19.704601+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":126,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.703696+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:19.703696+0000","last_peered":"2024-09-15T21:28:19.703696+0000","last_clean":"2024-09-15T21:28:19.703696+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:19.703696+0000","last_undegraded":"2024-09-15T21:28:19.703696+0000","last_fullsized":"2024-09-15T21:28:19.703696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24962,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704266+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:19.704266+0000","last_peered":"2024-09-15T21:28:19.704266+0000","last_clean":"2024-09-15T21:28:19.704266+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:19.704266+0000","last_undegraded":"2024-09-15T21:28:19.704266+0000","last_fullsized":"2024-09-15T21:28:19.704266+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25393,"reported_epoch":56,"state":"peering","last_fresh":"2024-09-15T21:28:19.670949+0000","last_change":"2024-09-15T21:28:19.712116+0000","last_active":"2024-09-15T21:28:19.712116+0000","last_peered":"2024-09-15T21:28:19.670949+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:19.670949+0000","last_undegraded":"2024-09-15T21:28:19.670949+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19654,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.275488+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:20.275488+0000","last_peered":"2024-09-15T21:28:20.275488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:20.275488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704820+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:19.704820+0000","last_peered":"2024-09-15T21:28:19.704820+0000","last_clean":"2024-09-15T21:28:19.704820+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:19.704820+0000","last_undegraded":"2024-09-15T21:28:19.704820+0000","last_fullsized":"2024-09-15T21:28:19.704820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19053,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.705383+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:19.705383+0000","last_peered":"2024-09-15T21:28:19.705383+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:19.705383+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25208,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490868,"kb_used_data":53856,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880972,"statfs":{"total":96636764160,"available":96134115328,"internally_reserved":0,"allocated":55148544,"data_stored":54824637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:23.688 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:23.688 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19219, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:19.703821+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:19.703821+0000', 'last_peered': '2024-09-15T21:28:19.703821+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:19.703821+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:23.689 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:23.689 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19666, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:19.703750+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:19.703750+0000', 'last_peered': '2024-09-15T21:28:19.703750+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:19.703750+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:23.689 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:23.689 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18834, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:19.704601+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:19.704601+0000', 'last_peered': '2024-09-15T21:28:19.704601+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:19.704601+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:23.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:28:23.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "54'18548", 'reported_seq': 25393, 'reported_epoch': 56, 'state': 'peering', 'last_fresh': '2024-09-15T21:28:19.670949+0000', 'last_change': '2024-09-15T21:28:19.712116+0000', 'last_active': '2024-09-15T21:28:19.712116+0000', 'last_peered': '2024-09-15T21:28:19.670949+0000', 'last_clean': '2024-09-15T21:18:41.344494+0000', 'last_became_active': '2024-09-15T21:18:43.846930+0000', 'last_became_peered': '2024-09-15T21:18:43.846930+0000', 'last_unstale': '2024-09-15T21:28:19.670949+0000', 'last_undegraded': '2024-09-15T21:28:19.670949+0000', 'last_fullsized': '2024-09-15T21:18:43.368584+0000', 'mapping_epoch': 57, 'log_start': "46'8500", 'ondisk_log_start': "46'8500", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'1630", 'last_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:10.504173+0000', 'objects_scrubbed': 1595, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 739, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16947, 'num_write_kb': 30704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3352, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:23.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:23.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 19654, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.275488+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:20.275488+0000', 'last_peered': '2024-09-15T21:28:20.275488+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:20.275488+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3039, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 395, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:23.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:23.691 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19053, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:19.705383+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:19.705383+0000', 'last_peered': '2024-09-15T21:28:19.705383+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:19.705383+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:23.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:24.097 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:24.097 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15357/6 objects degraded (255950.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":717,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1382240256,"bytes_avail":288528052224,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":15357,"degraded_total":6,"degraded_ratio":2559.5,"recovering_objects_per_sec":156,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1251,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:24.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:24.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:24.423 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:24.424 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:24.444 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-15T21:28:22.614190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25209,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1349844,"kb_used_data":163612,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281765676,"statfs":{"total":289910292480,"available":288528052224,"internally_reserved":0,"allocated":167538688,"data_stored":166539114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002245"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25355,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.710690+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:19.710690+0000","last_peered":"2024-09-15T21:28:19.710690+0000","last_clean":"2024-09-15T21:28:19.710690+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:19.710690+0000","last_undegraded":"2024-09-15T21:28:19.710690+0000","last_fullsized":"2024-09-15T21:28:19.710690+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18761,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.705716+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:19.705716+0000","last_peered":"2024-09-15T21:28:19.705716+0000","last_clean":"2024-09-15T21:28:19.705716+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:19.705716+0000","last_undegraded":"2024-09-15T21:28:19.705716+0000","last_fullsized":"2024-09-15T21:28:19.705716+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18664,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.707205+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:19.707205+0000","last_peered":"2024-09-15T21:28:19.707205+0000","last_clean":"2024-09-15T21:28:19.707205+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:19.707205+0000","last_undegraded":"2024-09-15T21:28:19.707205+0000","last_fullsized":"2024-09-15T21:28:19.707205+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18866,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704465+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:19.704465+0000","last_peered":"2024-09-15T21:28:19.704465+0000","last_clean":"2024-09-15T21:28:19.704465+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:19.704465+0000","last_undegraded":"2024-09-15T21:28:19.704465+0000","last_fullsized":"2024-09-15T21:28:19.704465+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19219,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703821+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:19.703821+0000","last_peered":"2024-09-15T21:28:19.703821+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:19.703821+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19666,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.703750+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:19.703750+0000","last_peered":"2024-09-15T21:28:19.703750+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:19.703750+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18549,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704347+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:19.704347+0000","last_peered":"2024-09-15T21:28:19.704347+0000","last_clean":"2024-09-15T21:28:19.704347+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:19.704347+0000","last_undegraded":"2024-09-15T21:28:19.704347+0000","last_fullsized":"2024-09-15T21:28:19.704347+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704186+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:19.704186+0000","last_peered":"2024-09-15T21:28:19.704186+0000","last_clean":"2024-09-15T21:28:19.704186+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:19.704186+0000","last_undegraded":"2024-09-15T21:28:19.704186+0000","last_fullsized":"2024-09-15T21:28:19.704186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18834,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.704601+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:19.704601+0000","last_peered":"2024-09-15T21:28:19.704601+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:19.704601+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":126,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.703696+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:19.703696+0000","last_peered":"2024-09-15T21:28:19.703696+0000","last_clean":"2024-09-15T21:28:19.703696+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:19.703696+0000","last_undegraded":"2024-09-15T21:28:19.703696+0000","last_fullsized":"2024-09-15T21:28:19.703696+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24962,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704266+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:19.704266+0000","last_peered":"2024-09-15T21:28:19.704266+0000","last_clean":"2024-09-15T21:28:19.704266+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:19.704266+0000","last_undegraded":"2024-09-15T21:28:19.704266+0000","last_fullsized":"2024-09-15T21:28:19.704266+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25393,"reported_epoch":56,"state":"peering","last_fresh":"2024-09-15T21:28:19.670949+0000","last_change":"2024-09-15T21:28:19.712116+0000","last_active":"2024-09-15T21:28:19.712116+0000","last_peered":"2024-09-15T21:28:19.670949+0000","last_clean":"2024-09-15T21:18:41.344494+0000","last_became_active":"2024-09-15T21:18:43.846930+0000","last_became_peered":"2024-09-15T21:18:43.846930+0000","last_unstale":"2024-09-15T21:28:19.670949+0000","last_undegraded":"2024-09-15T21:28:19.670949+0000","last_fullsized":"2024-09-15T21:18:43.368584+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19654,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.275488+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:20.275488+0000","last_peered":"2024-09-15T21:28:20.275488+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:20.275488+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.704820+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:19.704820+0000","last_peered":"2024-09-15T21:28:19.704820+0000","last_clean":"2024-09-15T21:28:19.704820+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:19.704820+0000","last_undegraded":"2024-09-15T21:28:19.704820+0000","last_fullsized":"2024-09-15T21:28:19.704820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19053,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.705383+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:19.705383+0000","last_peered":"2024-09-15T21:28:19.705383+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:19.705383+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25208,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953607,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411664,"kb_used_data":109244,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93960176,"statfs":{"total":96636764160,"available":96215220224,"internally_reserved":0,"allocated":111865856,"data_stored":111547286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490868,"kb_used_data":53856,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880972,"statfs":{"total":96636764160,"available":96134115328,"internally_reserved":0,"allocated":55148544,"data_stored":54824637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626552,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:27.447 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:27.804 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:27.821 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-15T21:28:26.615595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25959,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002447"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":21156,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:25.269773+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:25.269773+0000","last_peered":"2024-09-15T21:28:25.269773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:25.269773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25958,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:27.823 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:28.143 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:28.159 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-15T21:28:26.615595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25959,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002447"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":21156,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:25.269773+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:25.269773+0000","last_peered":"2024-09-15T21:28:25.269773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:25.269773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25958,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 21156, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:25.269773+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:25.269773+0000', 'last_peered': '2024-09-15T21:28:25.269773+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:25.269773+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2289, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1145, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2289}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:28.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:28.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:28.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:28.541 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:28.541 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14607/6 objects degraded (243450.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":721,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1353195520,"bytes_avail":288557096960,"bytes_total":289910292480,"degraded_objects":14607,"degraded_total":6,"degraded_ratio":2434.5,"recovering_objects_per_sec":181,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:28.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:28.560 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:28.890 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:28.890 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:28.908 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-15T21:28:28.616143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25959,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.913357"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":21156,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:25.269773+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:25.269773+0000","last_peered":"2024-09-15T21:28:25.269773+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:25.269773+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25958,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:31.911 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:32.249 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:32.249 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:32.267 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-15T21:28:30.616669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003059"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":22362,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:30.265584+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:30.265584+0000","last_peered":"2024-09-15T21:28:30.265584+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:30.265584+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1748,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626553,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:32.269 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:32.629 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:32.650 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-15T21:28:32.617317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1309000,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281806520,"statfs":{"total":289910292480,"available":288569876480,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.891919"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":22362,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:30.265584+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:30.265584+0000","last_peered":"2024-09-15T21:28:30.265584+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:30.265584+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1748,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443152,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928688,"statfs":{"total":96636764160,"available":96182976512,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:32.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 22362, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:30.265584+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:30.265584+0000', 'last_peered': '2024-09-15T21:28:30.265584+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:30.265584+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1686, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1748, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1686}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:32.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:32.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:32.653 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:33.041 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:33.042 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14004/6 objects degraded (233400.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":726,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1340416000,"bytes_avail":288569876480,"bytes_total":289910292480,"degraded_objects":14004,"degraded_total":6,"degraded_ratio":2334,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:33.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:33.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:33.379 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:33.380 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:33.397 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-15T21:28:32.617317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1309000,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281806520,"statfs":{"total":289910292480,"available":288569876480,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1214630922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1353,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.891919"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18868,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726685+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726685+0000","last_peered":"2024-09-15T21:28:20.726685+0000","last_clean":"2024-09-15T21:28:20.726685+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726685+0000","last_undegraded":"2024-09-15T21:28:20.726685+0000","last_fullsized":"2024-09-15T21:28:20.726685+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":22362,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:30.265584+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:30.265584+0000","last_peered":"2024-09-15T21:28:30.265584+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:30.265584+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1748,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1686}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953609,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149835,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443152,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93928688,"statfs":{"total":96636764160,"available":96182976512,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":453242540},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:36.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:36.748 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:36.748 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:36.766 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-15T21:28:36.618343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27008,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":23254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:35.273445+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:35.273445+0000","last_peered":"2024-09-15T21:28:35.273445+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:35.273445+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27007,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:36.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:37.091 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:37.092 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:37.111 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-15T21:28:36.618343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27008,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":23254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:35.273445+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:35.273445+0000","last_peered":"2024-09-15T21:28:35.273445+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:35.273445+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27007,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:37.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 23254, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:35.273445+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:35.273445+0000', 'last_peered': '2024-09-15T21:28:35.273445+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:35.273445+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1240, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2194, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:37.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.114 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:37.497 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:37.497 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13558/6 objects degraded (225966.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":730,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1344675840,"bytes_avail":288565616640,"bytes_total":289910292480,"degraded_objects":13558,"degraded_total":6,"degraded_ratio":2259.6666666666665,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1799,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:37.516 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:37.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:37.854 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:37.854 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:37.873 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-15T21:28:36.618343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27008,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1799,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25357,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728823+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728823+0000","last_peered":"2024-09-15T21:28:20.728823+0000","last_clean":"2024-09-15T21:28:20.728823+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728823+0000","last_undegraded":"2024-09-15T21:28:20.728823+0000","last_fullsized":"2024-09-15T21:28:20.728823+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":23254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:35.273445+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:35.273445+0000","last_peered":"2024-09-15T21:28:35.273445+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:35.273445+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13558,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27007,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953610,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149836,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:40.877 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:41.200 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:41.201 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:41.222 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-15T21:28:40.619608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27508,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1549,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1549,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":24254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:40.276251+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:40.276251+0000","last_peered":"2024-09-15T21:28:40.276251+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:40.276251+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27507,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:41.224 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:41.573 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:41.574 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:41.593 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-15T21:28:40.619608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27508,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1549,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1549,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":24254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:40.276251+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:40.276251+0000","last_peered":"2024-09-15T21:28:40.276251+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:40.276251+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27507,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:41.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:41.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:41.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 24254, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:40.276251+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:40.276251+0000', 'last_peered': '2024-09-15T21:28:40.276251+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:40.276251+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2694, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:41.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:41.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:41.971 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13058/6 objects degraded (217633.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":735,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1344675840,"bytes_avail":288565616640,"bytes_total":289910292480,"degraded_objects":13058,"degraded_total":6,"degraded_ratio":2176.3333333333335,"recovering_objects_per_sec":129,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1549,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:41.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:41.991 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:42.298 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:42.299 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:42.315 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-15T21:28:40.619608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27508,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313160,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281802360,"statfs":{"total":289910292480,"available":288565616640,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1549,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1549,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731562+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731562+0000","last_peered":"2024-09-15T21:28:20.731562+0000","last_clean":"2024-09-15T21:28:20.731562+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731562+0000","last_undegraded":"2024-09-15T21:28:20.731562+0000","last_fullsized":"2024-09-15T21:28:20.731562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":24254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:40.276251+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:40.276251+0000","last_peered":"2024-09-15T21:28:40.276251+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:40.276251+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27507,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447312,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924528,"statfs":{"total":96636764160,"available":96178716672,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:45.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:45.644 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:45.663 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-15T21:28:44.620987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27508,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1317320,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281798200,"statfs":{"total":289910292480,"available":288561356800,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-946,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003670"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":24254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:40.276251+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:40.276251+0000","last_peered":"2024-09-15T21:28:40.276251+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:40.276251+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27507,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451472,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920368,"statfs":{"total":96636764160,"available":96174456832,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:45.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:45.989 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:45.990 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:46.009 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-15T21:28:44.620987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27508,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1317320,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281798200,"statfs":{"total":289910292480,"available":288561356800,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1223150602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-946,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":946,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003670"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":24254,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:40.276251+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:40.276251+0000","last_peered":"2024-09-15T21:28:40.276251+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:40.276251+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2694,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27507,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124252160,"data_stored":124217370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149837,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626556,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451472,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93920368,"statfs":{"total":96636764160,"available":96174456832,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":461762220},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:46.010 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 24254, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:40.276251+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:40.276251+0000', 'last_peered': '2024-09-15T21:28:40.276251+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:40.276251+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 740, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2694, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:46.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:46.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:46.393 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13058/6 objects degraded (217633.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":739,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1348935680,"bytes_avail":288561356800,"bytes_total":289910292480,"degraded_objects":13058,"degraded_total":6,"degraded_ratio":2176.3333333333335,"recovering_objects_per_sec":78,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":946,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:46.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:46.413 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:46.735 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:46.736 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:46.754 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-15T21:28:46.621613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28122,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1560,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1560,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003818"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":25482,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:45.276837+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:45.276837+0000","last_peered":"2024-09-15T21:28:45.276837+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:45.276837+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28121,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124243968,"data_stored":124216136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953612,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:49.755 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:50.172 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:50.191 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-15T21:28:48.622244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28122,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002635"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":25482,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:45.276837+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:45.276837+0000","last_peered":"2024-09-15T21:28:45.276837+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:45.276837+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28121,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124243968,"data_stored":124216136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953612,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:50.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:50.593 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:50.594 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:50.613 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-15T21:28:48.622244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28122,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321480,"kb_used_data":122764,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281794040,"statfs":{"total":289910292480,"available":288557096960,"internally_reserved":0,"allocated":125710336,"data_stored":124702265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002635"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25358,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.728879+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:20.728879+0000","last_peered":"2024-09-15T21:28:20.728879+0000","last_clean":"2024-09-15T21:28:20.728879+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:20.728879+0000","last_undegraded":"2024-09-15T21:28:20.728879+0000","last_fullsized":"2024-09-15T21:28:20.728879+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18763,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725553+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:20.725553+0000","last_peered":"2024-09-15T21:28:20.725553+0000","last_clean":"2024-09-15T21:28:20.725553+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:20.725553+0000","last_undegraded":"2024-09-15T21:28:20.725553+0000","last_fullsized":"2024-09-15T21:28:20.725553+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18666,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.724729+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:20.724729+0000","last_peered":"2024-09-15T21:28:20.724729+0000","last_clean":"2024-09-15T21:28:20.724729+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:20.724729+0000","last_undegraded":"2024-09-15T21:28:20.724729+0000","last_fullsized":"2024-09-15T21:28:20.724729+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18869,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726732+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:20.726732+0000","last_peered":"2024-09-15T21:28:20.726732+0000","last_clean":"2024-09-15T21:28:20.726732+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:20.726732+0000","last_undegraded":"2024-09-15T21:28:20.726732+0000","last_fullsized":"2024-09-15T21:28:20.726732+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19222,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727919+0000","last_change":"2024-09-15T21:18:43.894658+0000","last_active":"2024-09-15T21:28:20.727919+0000","last_peered":"2024-09-15T21:28:20.727919+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:20.727919+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19668,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.726622+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:20.726622+0000","last_peered":"2024-09-15T21:28:20.726622+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:20.726622+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18551,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726573+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:20.726573+0000","last_peered":"2024-09-15T21:28:20.726573+0000","last_clean":"2024-09-15T21:28:20.726573+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:20.726573+0000","last_undegraded":"2024-09-15T21:28:20.726573+0000","last_fullsized":"2024-09-15T21:28:20.726573+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726393+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:20.726393+0000","last_peered":"2024-09-15T21:28:20.726393+0000","last_clean":"2024-09-15T21:28:20.726393+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:20.726393+0000","last_undegraded":"2024-09-15T21:28:20.726393+0000","last_fullsized":"2024-09-15T21:28:20.726393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18836,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.728447+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:20.728447+0000","last_peered":"2024-09-15T21:28:20.728447+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:20.728447+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":128,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726979+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:20.726979+0000","last_peered":"2024-09-15T21:28:20.726979+0000","last_clean":"2024-09-15T21:28:20.726979+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:20.726979+0000","last_undegraded":"2024-09-15T21:28:20.726979+0000","last_fullsized":"2024-09-15T21:28:20.726979+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24964,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.726845+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:20.726845+0000","last_peered":"2024-09-15T21:28:20.726845+0000","last_clean":"2024-09-15T21:28:20.726845+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:20.726845+0000","last_undegraded":"2024-09-15T21:28:20.726845+0000","last_fullsized":"2024-09-15T21:28:20.726845+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25402,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.795332+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:20.795332+0000","last_peered":"2024-09-15T21:28:20.795332+0000","last_clean":"2024-09-15T21:28:20.795332+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:20.795332+0000","last_undegraded":"2024-09-15T21:28:20.795332+0000","last_fullsized":"2024-09-15T21:28:20.795332+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":25482,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:45.276837+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:45.276837+0000","last_peered":"2024-09-15T21:28:45.276837+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:45.276837+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.725066+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:20.725066+0000","last_peered":"2024-09-15T21:28:20.725066+0000","last_clean":"2024-09-15T21:28:20.725066+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:20.725066+0000","last_undegraded":"2024-09-15T21:28:20.725066+0000","last_fullsized":"2024-09-15T21:28:20.725066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19055,"reported_epoch":58,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:20.727291+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:20.727291+0000","last_peered":"2024-09-15T21:28:20.727291+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:20.727291+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28121,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":124243968,"data_stored":124216136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953612,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374968,"kb_used_data":68388,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93996872,"statfs":{"total":96636764160,"available":96252796928,"internally_reserved":0,"allocated":70029312,"data_stored":69707603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149838,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490880,"kb_used_data":53864,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880960,"statfs":{"total":96636764160,"available":96134103040,"internally_reserved":0,"allocated":55156736,"data_stored":54827471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69545984,"data_stored":69545984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 19222, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727919+0000', 'last_change': '2024-09-15T21:18:43.894658+0000', 'last_active': '2024-09-15T21:28:20.727919+0000', 'last_peered': '2024-09-15T21:28:20.727919+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:20.727919+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3031, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 342, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19668, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.726622+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:20.726622+0000', 'last_peered': '2024-09-15T21:28:20.726622+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:20.726622+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18836, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.728447+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:20.728447+0000', 'last_peered': '2024-09-15T21:28:20.728447+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:20.728447+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "54'18710", 'reported_seq': 25482, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:45.276837+0000', 'last_change': '2024-09-15T21:28:19.672276+0000', 'last_active': '2024-09-15T21:28:45.276837+0000', 'last_peered': '2024-09-15T21:28:45.276837+0000', 'last_clean': '2024-09-15T21:18:41.355508+0000', 'last_became_active': '2024-09-15T21:18:43.883705+0000', 'last_became_peered': '2024-09-15T21:18:43.883705+0000', 'last_unstale': '2024-09-15T21:28:45.276837+0000', 'last_undegraded': '2024-09-15T21:18:43.486945+0000', 'last_fullsized': '2024-09-15T21:18:43.373231+0000', 'mapping_epoch': 43, 'log_start': "46'8700", 'ondisk_log_start': "46'8700", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2736", 'last_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.850672+0000', 'objects_scrubbed': 2691, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1139, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 126, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16017, 'num_write_kb': 26652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3308, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19055, 'reported_epoch': 58, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:20.727291+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:20.727291+0000', 'last_peered': '2024-09-15T21:28:20.727291+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:20.727291+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:50.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:51.015 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:51.015 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11824/6 objects degraded (197066.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":744,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1339166720,"bytes_avail":288571125760,"bytes_total":289910292480,"degraded_objects":11824,"degraded_total":6,"degraded_ratio":1970.666666666667,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:51.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:51.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:51.376 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:51.377 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:51.397 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-15T21:28:50.622577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28742,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1307780,"kb_used_data":109064,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281807740,"statfs":{"total":289910292480,"available":288571125760,"internally_reserved":0,"allocated":111681536,"data_stored":110651975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1227410442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002275"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18767,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896300+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896300+0000","last_peered":"2024-09-15T21:28:47.896300+0000","last_clean":"2024-09-15T21:28:47.896300+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896300+0000","last_undegraded":"2024-09-15T21:28:47.896300+0000","last_fullsized":"2024-09-15T21:28:47.896300+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25813,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731848+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:20.731848+0000","last_peered":"2024-09-15T21:28:20.731848+0000","last_clean":"2024-09-15T21:28:20.731848+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:20.731848+0000","last_undegraded":"2024-09-15T21:28:20.731848+0000","last_fullsized":"2024-09-15T21:28:20.731848+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18670,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895948+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895948+0000","last_peered":"2024-09-15T21:28:47.895948+0000","last_clean":"2024-09-15T21:28:47.895948+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895948+0000","last_undegraded":"2024-09-15T21:28:47.895948+0000","last_fullsized":"2024-09-15T21:28:47.895948+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":20467,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:50.277699+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:50.277699+0000","last_peered":"2024-09-15T21:28:50.277699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:50.277699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731618+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:20.731618+0000","last_peered":"2024-09-15T21:28:20.731618+0000","last_clean":"2024-09-15T21:28:20.731618+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:20.731618+0000","last_undegraded":"2024-09-15T21:28:20.731618+0000","last_fullsized":"2024-09-15T21:28:20.731618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25216,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731857+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:20.731857+0000","last_peered":"2024-09-15T21:28:20.731857+0000","last_clean":"2024-09-15T21:28:20.731857+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:20.731857+0000","last_undegraded":"2024-09-15T21:28:20.731857+0000","last_fullsized":"2024-09-15T21:28:20.731857+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":106,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731707+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:20.731707+0000","last_peered":"2024-09-15T21:28:20.731707+0000","last_clean":"2024-09-15T21:28:20.731707+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:20.731707+0000","last_undegraded":"2024-09-15T21:28:20.731707+0000","last_fullsized":"2024-09-15T21:28:20.731707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":25482,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:45.276837+0000","last_change":"2024-09-15T21:28:19.672276+0000","last_active":"2024-09-15T21:28:45.276837+0000","last_peered":"2024-09-15T21:28:45.276837+0000","last_clean":"2024-09-15T21:18:41.355508+0000","last_became_active":"2024-09-15T21:18:43.883705+0000","last_became_peered":"2024-09-15T21:18:43.883705+0000","last_unstale":"2024-09-15T21:28:45.276837+0000","last_undegraded":"2024-09-15T21:18:43.486945+0000","last_fullsized":"2024-09-15T21:18:43.373231+0000","mapping_epoch":43,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731840+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:20.731840+0000","last_peered":"2024-09-15T21:28:20.731840+0000","last_clean":"2024-09-15T21:28:20.731840+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:20.731840+0000","last_undegraded":"2024-09-15T21:28:20.731840+0000","last_fullsized":"2024-09-15T21:28:20.731840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25343,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732014+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:20.732014+0000","last_peered":"2024-09-15T21:28:20.732014+0000","last_clean":"2024-09-15T21:28:20.732014+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:20.732014+0000","last_undegraded":"2024-09-15T21:28:20.732014+0000","last_fullsized":"2024-09-15T21:28:20.732014+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731913+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:20.731913+0000","last_peered":"2024-09-15T21:28:20.731913+0000","last_clean":"2024-09-15T21:28:20.731913+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:20.731913+0000","last_undegraded":"2024-09-15T21:28:20.731913+0000","last_fullsized":"2024-09-15T21:28:20.731913+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.732089+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:20.732089+0000","last_peered":"2024-09-15T21:28:20.732089+0000","last_clean":"2024-09-15T21:28:20.732089+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:20.732089+0000","last_undegraded":"2024-09-15T21:28:20.732089+0000","last_fullsized":"2024-09-15T21:28:20.732089+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25730,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:28:20.731739+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:20.731739+0000","last_peered":"2024-09-15T21:28:20.731739+0000","last_clean":"2024-09-15T21:28:20.731739+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:20.731739+0000","last_undegraded":"2024-09-15T21:28:20.731739+0000","last_fullsized":"2024-09-15T21:28:20.731739+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28741,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626557,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455632,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93916208,"statfs":{"total":96636764160,"available":96170196992,"internally_reserved":0,"allocated":524288,"data_stored":167191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":466022060},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:54.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:54.734 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:54.734 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:54.752 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-15T21:28:54.623792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28868,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299468,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281816052,"statfs":{"total":289910292480,"available":288579637248,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":746,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002178"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18767,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896300+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896300+0000","last_peered":"2024-09-15T21:28:47.896300+0000","last_clean":"2024-09-15T21:28:47.896300+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896300+0000","last_undegraded":"2024-09-15T21:28:47.896300+0000","last_fullsized":"2024-09-15T21:28:47.896300+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18670,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895948+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895948+0000","last_peered":"2024-09-15T21:28:47.895948+0000","last_clean":"2024-09-15T21:28:47.895948+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895948+0000","last_undegraded":"2024-09-15T21:28:47.895948+0000","last_fullsized":"2024-09-15T21:28:47.895948+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":20467,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:50.277699+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:50.277699+0000","last_peered":"2024-09-15T21:28:50.277699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:50.277699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28867,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447320,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924520,"statfs":{"total":96636764160,"available":96178708480,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:54.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:55.142 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:55.143 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:55.160 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-15T21:28:54.623792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28868,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299468,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281816052,"statfs":{"total":289910292480,"available":288579637248,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":746,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002178"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18767,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896300+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896300+0000","last_peered":"2024-09-15T21:28:47.896300+0000","last_clean":"2024-09-15T21:28:47.896300+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896300+0000","last_undegraded":"2024-09-15T21:28:47.896300+0000","last_fullsized":"2024-09-15T21:28:47.896300+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18670,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895948+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895948+0000","last_peered":"2024-09-15T21:28:47.895948+0000","last_clean":"2024-09-15T21:28:47.895948+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895948+0000","last_undegraded":"2024-09-15T21:28:47.895948+0000","last_fullsized":"2024-09-15T21:28:47.895948+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":20467,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:50.277699+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:50.277699+0000","last_peered":"2024-09-15T21:28:50.277699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:50.277699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28867,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447320,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924520,"statfs":{"total":96636764160,"available":96178708480,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 20467, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:50.277699+0000', 'last_change': '2024-09-15T21:28:46.000603+0000', 'last_active': '2024-09-15T21:28:50.277699+0000', 'last_peered': '2024-09-15T21:28:50.277699+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:50.277699+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2411, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 962, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2411}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19672, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.893199+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:47.893199+0000', 'last_peered': '2024-09-15T21:28:47.893199+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:47.893199+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18840, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.899393+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:47.899393+0000', 'last_peered': '2024-09-15T21:28:47.899393+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:47.899393+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:55.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19059, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.896875+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:47.896875+0000', 'last_peered': '2024-09-15T21:28:47.896875+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:47.896875+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:55.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:55.570 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:55.570 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11698/6 objects degraded (194966.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":748,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1330655232,"bytes_avail":288579637248,"bytes_total":289910292480,"degraded_objects":11698,"degraded_total":6,"degraded_ratio":1949.666666666667,"recovering_objects_per_sec":93,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":746,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:55.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:55.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:55.934 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:55.934 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:55.952 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-15T21:28:54.623792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28868,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299468,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281816052,"statfs":{"total":289910292480,"available":288579637248,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1218890762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-746,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":746,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002178"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18767,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896300+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896300+0000","last_peered":"2024-09-15T21:28:47.896300+0000","last_clean":"2024-09-15T21:28:47.896300+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896300+0000","last_undegraded":"2024-09-15T21:28:47.896300+0000","last_fullsized":"2024-09-15T21:28:47.896300+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18670,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895948+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895948+0000","last_peered":"2024-09-15T21:28:47.895948+0000","last_clean":"2024-09-15T21:28:47.895948+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895948+0000","last_undegraded":"2024-09-15T21:28:47.895948+0000","last_fullsized":"2024-09-15T21:28:47.895948+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":20467,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:50.277699+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:50.277699+0000","last_peered":"2024-09-15T21:28:50.277699+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:50.277699+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11698,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28867,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953613,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149839,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447320,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93924520,"statfs":{"total":96636764160,"available":96178708480,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":457502380},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:58.953 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:59.301 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:59.321 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-15T21:28:58.625042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29419,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.732568"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":21569,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:55.274941+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:55.274941+0000","last_peered":"2024-09-15T21:28:55.274941+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:55.274941+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1513,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29418,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953614,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:59.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:59.654 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:28:59.654 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:28:59.674 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-15T21:28:58.625042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29419,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.732568"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":21569,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:55.274941+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:55.274941+0000","last_peered":"2024-09-15T21:28:55.274941+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:55.274941+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1513,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29418,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953614,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 21569, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:55.274941+0000', 'last_change': '2024-09-15T21:28:46.000603+0000', 'last_active': '2024-09-15T21:28:55.274941+0000', 'last_peered': '2024-09-15T21:28:55.274941+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:28:55.274941+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1860, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1513, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19672, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.893199+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:47.893199+0000', 'last_peered': '2024-09-15T21:28:47.893199+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:47.893199+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:59.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18840, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.899393+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:47.899393+0000', 'last_peered': '2024-09-15T21:28:47.899393+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:47.899393+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:28:59.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19059, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.896875+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:47.896875+0000', 'last_peered': '2024-09-15T21:28:47.896875+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:47.896875+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.677 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:00.064 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:00.064 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11147/6 objects degraded (185783.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":753,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1353396224,"bytes_avail":288556896256,"bytes_total":289910292480,"degraded_objects":11147,"degraded_total":6,"degraded_ratio":1857.833333333333,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:00.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:00.084 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:00.421 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:00.421 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:00.440 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-15T21:28:58.625042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29419,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.732568"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":21569,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:55.274941+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:28:55.274941+0000","last_peered":"2024-09-15T21:28:55.274941+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:28:55.274941+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1513,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29418,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953614,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149840,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:03.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:03.793 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:03.811 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-15T21:29:02.626372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29866,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003794"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":22463,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:00.280712+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:00.280712+0000","last_peered":"2024-09-15T21:29:00.280712+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:00.280712+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29865,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:03.813 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:04.120 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:04.120 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:04.140 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-15T21:29:02.626372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29866,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1124,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003794"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":22463,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:00.280712+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:00.280712+0000","last_peered":"2024-09-15T21:29:00.280712+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:00.280712+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29865,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:04.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 22463, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:00.280712+0000', 'last_change': '2024-09-15T21:28:46.000603+0000', 'last_active': '2024-09-15T21:29:00.280712+0000', 'last_peered': '2024-09-15T21:29:00.280712+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:29:00.280712+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1413, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1960, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1413}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19672, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.893199+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:47.893199+0000', 'last_peered': '2024-09-15T21:28:47.893199+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:47.893199+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18840, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.899393+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:47.899393+0000', 'last_peered': '2024-09-15T21:28:47.899393+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:47.899393+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:04.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19059, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.896875+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:47.896875+0000', 'last_peered': '2024-09-15T21:28:47.896875+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:47.896875+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.143 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:04.507 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:04.507 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10700/6 objects degraded (178333.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":757,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1353396224,"bytes_avail":288556896256,"bytes_total":289910292480,"degraded_objects":10700,"degraded_total":6,"degraded_ratio":1783.333333333333,"recovering_objects_per_sec":93,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1124,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:04.525 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:04.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:04.847 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:04.864 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-15T21:29:04.626962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29866,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321676,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793844,"statfs":{"total":289910292480,"available":288556896256,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003896"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":22463,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:00.280712+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:00.280712+0000","last_peered":"2024-09-15T21:29:00.280712+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:00.280712+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":110,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898782+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898782+0000","last_peered":"2024-09-15T21:28:47.898782+0000","last_clean":"2024-09-15T21:28:47.898782+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898782+0000","last_undegraded":"2024-09-15T21:28:47.898782+0000","last_fullsized":"2024-09-15T21:28:47.898782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899727+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899727+0000","last_peered":"2024-09-15T21:28:47.899727+0000","last_clean":"2024-09-15T21:28:47.899727+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899727+0000","last_undegraded":"2024-09-15T21:28:47.899727+0000","last_fullsized":"2024-09-15T21:28:47.899727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29865,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149841,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626560,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469528,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902312,"statfs":{"total":96636764160,"available":96155967488,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:07.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:08.196 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:08.214 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-15T21:29:06.627406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30332,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1325836,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281789684,"statfs":{"total":289910292480,"available":288552636416,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003614"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":23395,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:05.275150+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:05.275150+0000","last_peered":"2024-09-15T21:29:05.275150+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:05.275150+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-947}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30331,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473688,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898152,"statfs":{"total":96636764160,"available":96151707648,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:08.216 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:08.548 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:08.567 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-15T21:29:06.627406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30332,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1325836,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281789684,"statfs":{"total":289910292480,"available":288552636416,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003614"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":23395,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:05.275150+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:05.275150+0000","last_peered":"2024-09-15T21:29:05.275150+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:05.275150+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-947}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30331,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473688,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898152,"statfs":{"total":96636764160,"available":96151707648,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:08.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:08.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 23395, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:05.275150+0000', 'last_change': '2024-09-15T21:28:46.000603+0000', 'last_active': '2024-09-15T21:29:05.275150+0000', 'last_peered': '2024-09-15T21:29:05.275150+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:29:05.275150+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 947, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2426, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -947}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:08.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:08.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19672, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.893199+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:47.893199+0000', 'last_peered': '2024-09-15T21:28:47.893199+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:47.893199+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:08.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:08.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18840, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.899393+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:47.899393+0000', 'last_peered': '2024-09-15T21:28:47.899393+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:47.899393+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:08.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:08.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19059, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.896875+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:47.896875+0000', 'last_peered': '2024-09-15T21:28:47.896875+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:47.896875+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:08.569 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:08.956 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10234/6 objects degraded (170566.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1357656064,"bytes_avail":288552636416,"bytes_total":289910292480,"degraded_objects":10234,"degraded_total":6,"degraded_ratio":1705.666666666667,"recovering_objects_per_sec":76,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":913,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:08.975 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:08.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:09.290 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:09.291 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:09.311 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-15T21:29:08.628394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30332,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1325836,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281789684,"statfs":{"total":289910292480,"available":288552636416,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004175"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":23395,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:05.275150+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:05.275150+0000","last_peered":"2024-09-15T21:29:05.275150+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:05.275150+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-947}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30331,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953617,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149842,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473688,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898152,"statfs":{"total":96636764160,"available":96151707648,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:12.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:12.651 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:12.652 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:12.671 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-15T21:29:12.629695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30921,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313356,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281802164,"statfs":{"total":289910292480,"available":288565415936,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003992"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":24573,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:10.283356+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:10.283356+0000","last_peered":"2024-09-15T21:29:10.283356+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:10.283356+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30920,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461208,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910632,"statfs":{"total":96636764160,"available":96164487168,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:12.672 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:12.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:12.992 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:13.011 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-15T21:29:12.629695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30921,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313356,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281802164,"statfs":{"total":289910292480,"available":288565415936,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003992"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":24573,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:10.283356+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:10.283356+0000","last_peered":"2024-09-15T21:29:10.283356+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:10.283356+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30920,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461208,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910632,"statfs":{"total":96636764160,"available":96164487168,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "54'18381", 'reported_seq': 24573, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:10.283356+0000', 'last_change': '2024-09-15T21:28:46.000603+0000', 'last_active': '2024-09-15T21:29:10.283356+0000', 'last_peered': '2024-09-15T21:29:10.283356+0000', 'last_clean': '2024-09-15T21:18:41.310391+0000', 'last_became_active': '2024-09-15T21:18:43.883328+0000', 'last_became_peered': '2024-09-15T21:18:43.883328+0000', 'last_unstale': '2024-09-15T21:29:10.283356+0000', 'last_undegraded': '2024-09-15T21:18:43.501324+0000', 'last_fullsized': '2024-09-15T21:18:43.374553+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2630", 'last_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:26.026796+0000', 'objects_scrubbed': 2570, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1295, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 358, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3015, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -358}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19672, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.893199+0000', 'last_change': '2024-09-15T21:18:43.898172+0000', 'last_active': '2024-09-15T21:28:47.893199+0000', 'last_peered': '2024-09-15T21:28:47.893199+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:28:47.893199+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3166, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 307, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18840, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.899393+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:28:47.899393+0000', 'last_peered': '2024-09-15T21:28:47.899393+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:28:47.899393+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:13.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:13.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19059, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:47.896875+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:28:47.896875+0000', 'last_peered': '2024-09-15T21:28:47.896875+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:28:47.896875+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:13.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:13.414 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:13.414 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9645/6 objects degraded (160750.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":766,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1344876544,"bytes_avail":288565415936,"bytes_total":289910292480,"degraded_objects":9645,"degraded_total":6,"degraded_ratio":1607.5,"recovering_objects_per_sec":87,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1055,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:13.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:13.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:13.777 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:13.777 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:13.794 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-15T21:29:12.629695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30921,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1313356,"kb_used_data":109072,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281802164,"statfs":{"total":289910292480,"available":288565415936,"internally_reserved":0,"allocated":111689728,"data_stored":110656380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003992"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.900327+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:28:47.900327+0000","last_peered":"2024-09-15T21:28:47.900327+0000","last_clean":"2024-09-15T21:28:47.900327+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:28:47.900327+0000","last_undegraded":"2024-09-15T21:28:47.900327+0000","last_fullsized":"2024-09-15T21:28:47.900327+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18768,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896371+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:28:47.896371+0000","last_peered":"2024-09-15T21:28:47.896371+0000","last_clean":"2024-09-15T21:28:47.896371+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:28:47.896371+0000","last_undegraded":"2024-09-15T21:28:47.896371+0000","last_fullsized":"2024-09-15T21:28:47.896371+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25817,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899016+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:28:47.899016+0000","last_peered":"2024-09-15T21:28:47.899016+0000","last_clean":"2024-09-15T21:28:47.899016+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:28:47.899016+0000","last_undegraded":"2024-09-15T21:28:47.899016+0000","last_fullsized":"2024-09-15T21:28:47.899016+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895998+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:28:47.895998+0000","last_peered":"2024-09-15T21:28:47.895998+0000","last_clean":"2024-09-15T21:28:47.895998+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:28:47.895998+0000","last_undegraded":"2024-09-15T21:28:47.895998+0000","last_fullsized":"2024-09-15T21:28:47.895998+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18873,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.893094+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:28:47.893094+0000","last_peered":"2024-09-15T21:28:47.893094+0000","last_clean":"2024-09-15T21:28:47.893094+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:28:47.893094+0000","last_undegraded":"2024-09-15T21:28:47.893094+0000","last_fullsized":"2024-09-15T21:28:47.893094+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":24573,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:10.283356+0000","last_change":"2024-09-15T21:28:46.000603+0000","last_active":"2024-09-15T21:29:10.283356+0000","last_peered":"2024-09-15T21:29:10.283356+0000","last_clean":"2024-09-15T21:18:41.310391+0000","last_became_active":"2024-09-15T21:18:43.883328+0000","last_became_peered":"2024-09-15T21:18:43.883328+0000","last_unstale":"2024-09-15T21:29:10.283356+0000","last_undegraded":"2024-09-15T21:18:43.501324+0000","last_fullsized":"2024-09-15T21:18:43.374553+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3015,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.893199+0000","last_change":"2024-09-15T21:18:43.898172+0000","last_active":"2024-09-15T21:28:47.893199+0000","last_peered":"2024-09-15T21:28:47.893199+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:28:47.893199+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18555,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892765+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:28:47.892765+0000","last_peered":"2024-09-15T21:28:47.892765+0000","last_clean":"2024-09-15T21:28:47.892765+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:28:47.892765+0000","last_undegraded":"2024-09-15T21:28:47.892765+0000","last_fullsized":"2024-09-15T21:28:47.892765+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898465+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:28:47.898465+0000","last_peered":"2024-09-15T21:28:47.898465+0000","last_clean":"2024-09-15T21:28:47.898465+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:28:47.898465+0000","last_undegraded":"2024-09-15T21:28:47.898465+0000","last_fullsized":"2024-09-15T21:28:47.898465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25220,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899297+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:28:47.899297+0000","last_peered":"2024-09-15T21:28:47.899297+0000","last_clean":"2024-09-15T21:28:47.899297+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:28:47.899297+0000","last_undegraded":"2024-09-15T21:28:47.899297+0000","last_fullsized":"2024-09-15T21:28:47.899297+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895529+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:28:47.895529+0000","last_peered":"2024-09-15T21:28:47.895529+0000","last_clean":"2024-09-15T21:28:47.895529+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:28:47.895529+0000","last_undegraded":"2024-09-15T21:28:47.895529+0000","last_fullsized":"2024-09-15T21:28:47.895529+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18840,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.899393+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:28:47.899393+0000","last_peered":"2024-09-15T21:28:47.899393+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:28:47.899393+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":132,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.892928+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:28:47.892928+0000","last_peered":"2024-09-15T21:28:47.892928+0000","last_clean":"2024-09-15T21:28:47.892928+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:28:47.892928+0000","last_undegraded":"2024-09-15T21:28:47.892928+0000","last_fullsized":"2024-09-15T21:28:47.892928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24968,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895676+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:28:47.895676+0000","last_peered":"2024-09-15T21:28:47.895676+0000","last_clean":"2024-09-15T21:28:47.895676+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:28:47.895676+0000","last_undegraded":"2024-09-15T21:28:47.895676+0000","last_fullsized":"2024-09-15T21:28:47.895676+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":111,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.898867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:28:47.898867+0000","last_peered":"2024-09-15T21:28:47.898867+0000","last_clean":"2024-09-15T21:28:47.898867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:28:47.898867+0000","last_undegraded":"2024-09-15T21:28:47.898867+0000","last_fullsized":"2024-09-15T21:28:47.898867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25406,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.895380+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:28:47.895380+0000","last_peered":"2024-09-15T21:28:47.895380+0000","last_clean":"2024-09-15T21:28:47.895380+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:28:47.895380+0000","last_undegraded":"2024-09-15T21:28:47.895380+0000","last_fullsized":"2024-09-15T21:28:47.895380+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19475,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.019694+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:28:48.019694+0000","last_peered":"2024-09-15T21:28:48.019694+0000","last_clean":"2024-09-15T21:28:48.019694+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:28:48.019694+0000","last_undegraded":"2024-09-15T21:28:48.019694+0000","last_fullsized":"2024-09-15T21:28:48.019694+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899777+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:28:47.899777+0000","last_peered":"2024-09-15T21:28:47.899777+0000","last_clean":"2024-09-15T21:28:47.899777+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:28:47.899777+0000","last_undegraded":"2024-09-15T21:28:47.899777+0000","last_fullsized":"2024-09-15T21:28:47.899777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25347,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899410+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:28:47.899410+0000","last_peered":"2024-09-15T21:28:47.899410+0000","last_clean":"2024-09-15T21:28:47.899410+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:28:47.899410+0000","last_undegraded":"2024-09-15T21:28:47.899410+0000","last_fullsized":"2024-09-15T21:28:47.899410+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.896213+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:28:47.896213+0000","last_peered":"2024-09-15T21:28:47.896213+0000","last_clean":"2024-09-15T21:28:47.896213+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:28:47.896213+0000","last_undegraded":"2024-09-15T21:28:47.896213+0000","last_fullsized":"2024-09-15T21:28:47.896213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19059,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:47.896875+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:28:47.896875+0000","last_peered":"2024-09-15T21:28:47.896875+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:28:47.896875+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899277+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:28:47.899277+0000","last_peered":"2024-09-15T21:28:47.899277+0000","last_clean":"2024-09-15T21:28:47.899277+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:28:47.899277+0000","last_undegraded":"2024-09-15T21:28:47.899277+0000","last_fullsized":"2024-09-15T21:28:47.899277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899438+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:28:47.899438+0000","last_peered":"2024-09-15T21:28:47.899438+0000","last_clean":"2024-09-15T21:28:47.899438+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:28:47.899438+0000","last_undegraded":"2024-09-15T21:28:47.899438+0000","last_fullsized":"2024-09-15T21:28:47.899438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25734,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:28:47.899595+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:28:47.899595+0000","last_peered":"2024-09-15T21:28:47.899595+0000","last_clean":"2024-09-15T21:28:47.899595+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:28:47.899595+0000","last_undegraded":"2024-09-15T21:28:47.899595+0000","last_fullsized":"2024-09-15T21:28:47.899595+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30920,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110182400,"data_stored":110154568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953618,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361252,"kb_used_data":54672,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010588,"statfs":{"total":96636764160,"available":96266842112,"internally_reserved":0,"allocated":55984128,"data_stored":55651674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490896,"kb_used_data":53880,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880944,"statfs":{"total":96636764160,"available":96134086656,"internally_reserved":0,"allocated":55173120,"data_stored":54833110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461208,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910632,"statfs":{"total":96636764160,"available":96164487168,"internally_reserved":0,"allocated":532480,"data_stored":171596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55484416,"data_stored":55484416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:16.796 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:17.146 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:17.146 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:17.169 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-15T21:29:16.630738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321708,"kb_used_data":109104,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793812,"statfs":{"total":289910292480,"available":288556863488,"internally_reserved":0,"allocated":111722496,"data_stored":110669129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002344"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":20244,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:15.279749+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:15.279749+0000","last_peered":"2024-09-15T21:29:15.279749+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:15.279749+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25351,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821540+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821540+0000","last_peered":"2024-09-15T21:29:14.821540+0000","last_clean":"2024-09-15T21:29:14.821540+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821540+0000","last_undegraded":"2024-09-15T21:29:14.821540+0000","last_fullsized":"2024-09-15T21:29:14.821540+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361260,"kb_used_data":54680,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010580,"statfs":{"total":96636764160,"available":96266833920,"internally_reserved":0,"allocated":55992320,"data_stored":55654479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:17.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:17.550 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:17.551 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:17.573 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-15T21:29:16.630738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321708,"kb_used_data":109104,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793812,"statfs":{"total":289910292480,"available":288556863488,"internally_reserved":0,"allocated":111722496,"data_stored":110669129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002344"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":20244,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:15.279749+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:15.279749+0000","last_peered":"2024-09-15T21:29:15.279749+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:15.279749+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25351,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821540+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821540+0000","last_peered":"2024-09-15T21:29:14.821540+0000","last_clean":"2024-09-15T21:29:14.821540+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821540+0000","last_undegraded":"2024-09-15T21:29:14.821540+0000","last_fullsized":"2024-09-15T21:29:14.821540+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361260,"kb_used_data":54680,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010580,"statfs":{"total":96636764160,"available":96266833920,"internally_reserved":0,"allocated":55992320,"data_stored":55654479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:17.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:17.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 20244, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:15.279749+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:15.279749+0000', 'last_peered': '2024-09-15T21:29:15.279749+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:15.279749+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2883, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 590, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2883}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:17.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:17.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:17.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:17.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:17.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:17.983 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:17.984 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9004/6 objects degraded (150066.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":771,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1353428992,"bytes_avail":288556863488,"bytes_total":289910292480,"degraded_objects":9004,"degraded_total":6,"degraded_ratio":1500.666666666667,"recovering_objects_per_sec":153,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1230,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:18.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:18.003 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:18.345 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:18.346 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:18.364 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-15T21:29:16.630738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31562,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1321708,"kb_used_data":109104,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281793812,"statfs":{"total":289910292480,"available":288556863488,"internally_reserved":0,"allocated":111722496,"data_stored":110669129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002344"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":20244,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:15.279749+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:15.279749+0000","last_peered":"2024-09-15T21:29:15.279749+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:15.279749+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2883}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25351,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821540+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821540+0000","last_peered":"2024-09-15T21:29:14.821540+0000","last_clean":"2024-09-15T21:29:14.821540+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821540+0000","last_undegraded":"2024-09-15T21:29:14.821540+0000","last_fullsized":"2024-09-15T21:29:14.821540+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31561,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953619,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361260,"kb_used_data":54680,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94010580,"statfs":{"total":96636764160,"available":96266833920,"internally_reserved":0,"allocated":55992320,"data_stored":55654479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149844,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:21.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:21.755 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:21.755 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:21.774 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-15T21:29:20.632213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32070,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308232,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281807288,"statfs":{"total":289910292480,"available":288570662912,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002518"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":21260,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:20.285598+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:20.285598+0000","last_peered":"2024-09-15T21:29:20.285598+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:20.285598+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25351,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821540+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821540+0000","last_peered":"2024-09-15T21:29:14.821540+0000","last_clean":"2024-09-15T21:29:14.821540+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821540+0000","last_undegraded":"2024-09-15T21:29:14.821540+0000","last_fullsized":"2024-09-15T21:29:14.821540+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32069,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:21.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:22.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:29:22.114+0000 7f70a37fe640 0 --2- 172.21.15.114:0/145127448 >> [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] conn(0x7f70a4093590 0x7f70a4093970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:29:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:22.308 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:22.328 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-15T21:29:20.632213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32070,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308232,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281807288,"statfs":{"total":289910292480,"available":288570662912,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002518"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":21260,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:20.285598+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:20.285598+0000","last_peered":"2024-09-15T21:29:20.285598+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:20.285598+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25351,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821540+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821540+0000","last_peered":"2024-09-15T21:29:14.821540+0000","last_clean":"2024-09-15T21:29:14.821540+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821540+0000","last_undegraded":"2024-09-15T21:29:14.821540+0000","last_fullsized":"2024-09-15T21:29:14.821540+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32069,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 21260, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:20.285598+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:20.285598+0000', 'last_peered': '2024-09-15T21:29:20.285598+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:20.285598+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2375, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1098, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:22.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:22.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:22.767 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8496/6 objects degraded (141600.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":775,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1339629568,"bytes_avail":288570662912,"bytes_total":289910292480,"degraded_objects":8496,"degraded_total":6,"degraded_ratio":1416,"recovering_objects_per_sec":143,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1149,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:22.789 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:22.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:23.104 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:23.105 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:23.125 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-15T21:29:22.632981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32070,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.824904"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":21260,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:20.285598+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:20.285598+0000","last_peered":"2024-09-15T21:29:20.285598+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:20.285598+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32069,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:26.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:26.453 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:26.454 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:26.472 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-15T21:29:24.633606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32070,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003435"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":21260,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:20.285598+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:20.285598+0000","last_peered":"2024-09-15T21:29:20.285598+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:20.285598+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8496,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32069,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149845,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:26.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:26.809 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:26.809 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:26.828 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-15T21:29:26.634115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32451,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.814384"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22022,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:25.286232+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:25.286232+0000","last_peered":"2024-09-15T21:29:25.286232+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:25.286232+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32450,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 22022, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:25.286232+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:25.286232+0000', 'last_peered': '2024-09-15T21:29:25.286232+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:25.286232+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1994, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1479, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1994}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:26.830 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:26.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:27.208 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8115/6 objects degraded (135250.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":780,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1343889408,"bytes_avail":288566403072,"bytes_total":289910292480,"degraded_objects":8115,"degraded_total":6,"degraded_ratio":1352.5,"recovering_objects_per_sec":129,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1530,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:27.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:27.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:27.607 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:27.625 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-15T21:29:26.634115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32451,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.814384"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22022,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:25.286232+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:25.286232+0000","last_peered":"2024-09-15T21:29:25.286232+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:25.286232+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32450,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953621,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149846,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:30.627 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:30.981 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:30.981 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:31.000 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:29:30.635060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003491"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:31.001 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:31.339 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:31.358 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:29:30.635060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003491"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 22848, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:30.284599+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:30.284599+0000', 'last_peered': '2024-09-15T21:29:30.284599+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:30.284599+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1581, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1892, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:31.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:31.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:31.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:31.744 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7702/6 objects degraded (128366.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":784,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1343889408,"bytes_avail":288566403072,"bytes_total":289910292480,"degraded_objects":7702,"degraded_total":6,"degraded_ratio":1283.6666666666667,"recovering_objects_per_sec":108,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:31.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:31.764 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:32.098 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:32.099 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:32.116 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:29:30.635060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312392,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803128,"statfs":{"total":289910292480,"available":288566403072,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003491"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821687+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821687+0000","last_peered":"2024-09-15T21:29:14.821687+0000","last_clean":"2024-09-15T21:29:14.821687+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821687+0000","last_undegraded":"2024-09-15T21:29:14.821687+0000","last_fullsized":"2024-09-15T21:29:14.821687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626565,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473696,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898144,"statfs":{"total":96636764160,"available":96151699456,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:35.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:35.451 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:35.452 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:35.470 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-15T21:29:34.636406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299912,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281815608,"statfs":{"total":289910292480,"available":288579182592,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461216,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910624,"statfs":{"total":96636764160,"available":96164478976,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:35.472 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:35.808 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:35.808 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:35.827 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-15T21:29:34.636406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299912,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281815608,"statfs":{"total":289910292480,"available":288579182592,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461216,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910624,"statfs":{"total":96636764160,"available":96164478976,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 22848, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:30.284599+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:30.284599+0000', 'last_peered': '2024-09-15T21:29:30.284599+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:30.284599+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1581, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1892, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:35.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:35.829 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:36.228 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7702/6 objects degraded (128366.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1331109888,"bytes_avail":288579182592,"bytes_total":289910292480,"degraded_objects":7702,"degraded_total":6,"degraded_ratio":1283.6666666666667,"recovering_objects_per_sec":66,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":794,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:36.250 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:36.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:36.582 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:36.583 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:36.603 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-15T21:29:34.636406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32864,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1299912,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281815608,"statfs":{"total":289910292480,"available":288579182592,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-794,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":22848,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:30.284599+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:30.284599+0000","last_peered":"2024-09-15T21:29:30.284599+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:30.284599+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1581,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32863,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953622,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149847,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461216,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910624,"statfs":{"total":96636764160,"available":96164478976,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:39.606 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:39.940 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:39.941 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:39.959 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-15T21:29:38.637477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33304,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1304072,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281811448,"statfs":{"total":289910292480,"available":288574922752,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003361"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":23728,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:35.293219+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:35.293219+0000","last_peered":"2024-09-15T21:29:35.293219+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:35.293219+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33303,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465376,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906464,"statfs":{"total":96636764160,"available":96160219136,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:39.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:40.275 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:40.276 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:40.295 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-15T21:29:38.637477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33304,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1304072,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281811448,"statfs":{"total":289910292480,"available":288574922752,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-853,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003361"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":23728,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:35.293219+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:35.293219+0000","last_peered":"2024-09-15T21:29:35.293219+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:35.293219+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1141}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25410,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823027+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823027+0000","last_peered":"2024-09-15T21:29:14.823027+0000","last_clean":"2024-09-15T21:29:14.823027+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823027+0000","last_undegraded":"2024-09-15T21:29:14.823027+0000","last_fullsized":"2024-09-15T21:29:14.823027+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33303,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953623,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149848,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465376,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906464,"statfs":{"total":96636764160,"available":96160219136,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 23728, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:35.293219+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:35.293219+0000', 'last_peered': '2024-09-15T21:29:35.293219+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:35.293219+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1141, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2332, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1141}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:40.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:40.297 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:40.672 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:40.672 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7262/6 objects degraded (121033.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1335369728,"bytes_avail":288574922752,"bytes_total":289910292480,"degraded_objects":7262,"degraded_total":6,"degraded_ratio":1210.3333333333333,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":853,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:40.692 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:40.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:41.026 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:41.026 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:41.044 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-15T21:29:40.638035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33858,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1304072,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281811448,"statfs":{"total":289910292480,"available":288574922752,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003548"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":24836,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:40.290139+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:40.290139+0000","last_peered":"2024-09-15T21:29:40.290139+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:40.290139+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33857,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465376,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906464,"statfs":{"total":96636764160,"available":96160219136,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:44.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:44.365 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:44.384 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-15T21:29:42.638665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33858,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308232,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281807288,"statfs":{"total":289910292480,"available":288570662912,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":994,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003604"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":24836,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:40.290139+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:40.290139+0000","last_peered":"2024-09-15T21:29:40.290139+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:40.290139+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33857,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:44.386 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:44.710 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:44.711 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:44.729 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-15T21:29:44.639235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33858,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308232,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281807288,"statfs":{"total":289910292480,"available":288570662912,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":994,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003480"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":24836,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:40.290139+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:40.290139+0000","last_peered":"2024-09-15T21:29:40.290139+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:40.290139+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33857,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:44.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:44.730 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 24836, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:40.290139+0000', 'last_change': '2024-09-15T21:29:12.843442+0000', 'last_active': '2024-09-15T21:29:40.290139+0000', 'last_peered': '2024-09-15T21:29:40.290139+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:40.290139+0000', 'last_undegraded': '2024-09-15T21:18:43.495967+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 43, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 587, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2886, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -587}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:44.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:44.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18844, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826971+0000', 'last_change': '2024-09-15T21:18:43.899638+0000', 'last_active': '2024-09-15T21:29:14.826971+0000', 'last_peered': '2024-09-15T21:29:14.826971+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:14.826971+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3015}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:44.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:44.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:44.731 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:45.108 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:45.108 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6708/6 objects degraded (111800.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":798,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1339629568,"bytes_avail":288570662912,"bytes_total":289910292480,"degraded_objects":6708,"degraded_total":6,"degraded_ratio":1118,"recovering_objects_per_sec":82,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":994,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:45.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:45.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:45.469 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:45.470 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:45.488 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-15T21:29:44.639235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33858,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308232,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281807288,"statfs":{"total":289910292480,"available":288570662912,"internally_reserved":0,"allocated":97923072,"data_stored":96864297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":994,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003480"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25821,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820849+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:14.820849+0000","last_peered":"2024-09-15T21:29:14.820849+0000","last_clean":"2024-09-15T21:29:14.820849+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:14.820849+0000","last_undegraded":"2024-09-15T21:29:14.820849+0000","last_fullsized":"2024-09-15T21:29:14.820849+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19223,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.007476+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:15.007476+0000","last_peered":"2024-09-15T21:29:15.007476+0000","last_clean":"2024-09-15T21:29:15.007476+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:15.007476+0000","last_undegraded":"2024-09-15T21:29:15.007476+0000","last_fullsized":"2024-09-15T21:29:15.007476+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":24836,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:40.290139+0000","last_change":"2024-09-15T21:29:12.843442+0000","last_active":"2024-09-15T21:29:40.290139+0000","last_peered":"2024-09-15T21:29:40.290139+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:40.290139+0000","last_undegraded":"2024-09-15T21:18:43.495967+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":43,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2886,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820128+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:14.820128+0000","last_peered":"2024-09-15T21:29:14.820128+0000","last_clean":"2024-09-15T21:29:14.820128+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:14.820128+0000","last_undegraded":"2024-09-15T21:29:14.820128+0000","last_fullsized":"2024-09-15T21:29:14.820128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25224,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820993+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:14.820993+0000","last_peered":"2024-09-15T21:29:14.820993+0000","last_clean":"2024-09-15T21:29:14.820993+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:14.820993+0000","last_undegraded":"2024-09-15T21:29:14.820993+0000","last_fullsized":"2024-09-15T21:29:14.820993+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18844,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826971+0000","last_change":"2024-09-15T21:18:43.899638+0000","last_active":"2024-09-15T21:29:14.826971+0000","last_peered":"2024-09-15T21:29:14.826971+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:14.826971+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3015}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.820867+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:14.820867+0000","last_peered":"2024-09-15T21:29:14.820867+0000","last_clean":"2024-09-15T21:29:14.820867+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:14.820867+0000","last_undegraded":"2024-09-15T21:29:14.820867+0000","last_fullsized":"2024-09-15T21:29:14.820867+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19479,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821805+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:14.821805+0000","last_peered":"2024-09-15T21:29:14.821805+0000","last_clean":"2024-09-15T21:29:14.821805+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:14.821805+0000","last_undegraded":"2024-09-15T21:29:14.821805+0000","last_fullsized":"2024-09-15T21:29:14.821805+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821258+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:14.821258+0000","last_peered":"2024-09-15T21:29:14.821258+0000","last_clean":"2024-09-15T21:29:14.821258+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:14.821258+0000","last_undegraded":"2024-09-15T21:29:14.821258+0000","last_fullsized":"2024-09-15T21:29:14.821258+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25352,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821655+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:14.821655+0000","last_peered":"2024-09-15T21:29:14.821655+0000","last_clean":"2024-09-15T21:29:14.821655+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:14.821655+0000","last_undegraded":"2024-09-15T21:29:14.821655+0000","last_fullsized":"2024-09-15T21:29:14.821655+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821415+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:14.821415+0000","last_peered":"2024-09-15T21:29:14.821415+0000","last_clean":"2024-09-15T21:29:14.821415+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:14.821415+0000","last_undegraded":"2024-09-15T21:29:14.821415+0000","last_fullsized":"2024-09-15T21:29:14.821415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821788+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:14.821788+0000","last_peered":"2024-09-15T21:29:14.821788+0000","last_clean":"2024-09-15T21:29:14.821788+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:14.821788+0000","last_undegraded":"2024-09-15T21:29:14.821788+0000","last_fullsized":"2024-09-15T21:29:14.821788+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25738,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.821140+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:14.821140+0000","last_peered":"2024-09-15T21:29:14.821140+0000","last_clean":"2024-09-15T21:29:14.821140+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:14.821140+0000","last_undegraded":"2024-09-15T21:29:14.821140+0000","last_fullsized":"2024-09-15T21:29:14.821140+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33857,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96366592,"data_stored":96345718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953624,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149849,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626568,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469536,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902304,"statfs":{"total":96636764160,"available":96155959296,"internally_reserved":0,"allocated":540672,"data_stored":175951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:48.491 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:48.910 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:48.910 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:48.934 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:29:48.640245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34477,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312396,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803124,"statfs":{"total":289910292480,"available":288566398976,"internally_reserved":0,"allocated":97923072,"data_stored":96865843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002209"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25823,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745572+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:45.745572+0000","last_peered":"2024-09-15T21:29:45.745572+0000","last_clean":"2024-09-15T21:29:45.745572+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:45.745572+0000","last_undegraded":"2024-09-15T21:29:45.745572+0000","last_fullsized":"2024-09-15T21:29:45.745572+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19225,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745190+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:45.745190+0000","last_peered":"2024-09-15T21:29:45.745190+0000","last_clean":"2024-09-15T21:29:45.745190+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:45.745190+0000","last_undegraded":"2024-09-15T21:29:45.745190+0000","last_fullsized":"2024-09-15T21:29:45.745190+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19663,"reported_epoch":63,"state":"peering","last_fresh":"2024-09-15T21:29:44.899174+0000","last_change":"2024-09-15T21:29:46.319113+0000","last_active":"2024-09-15T21:29:46.319113+0000","last_peered":"2024-09-15T21:29:44.899174+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:44.899174+0000","last_undegraded":"2024-09-15T21:29:44.899174+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.743935+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:45.743935+0000","last_peered":"2024-09-15T21:29:45.743935+0000","last_clean":"2024-09-15T21:29:45.743935+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:45.743935+0000","last_undegraded":"2024-09-15T21:29:45.743935+0000","last_fullsized":"2024-09-15T21:29:45.743935+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25226,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744775+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:45.744775+0000","last_peered":"2024-09-15T21:29:45.744775+0000","last_clean":"2024-09-15T21:29:45.744775+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:45.744775+0000","last_undegraded":"2024-09-15T21:29:45.744775+0000","last_fullsized":"2024-09-15T21:29:45.744775+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18909,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:45.295640+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:45.295640+0000","last_peered":"2024-09-15T21:29:45.295640+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:45.295640+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2985}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":117,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744447+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:45.744447+0000","last_peered":"2024-09-15T21:29:45.744447+0000","last_clean":"2024-09-15T21:29:45.744447+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:45.744447+0000","last_undegraded":"2024-09-15T21:29:45.744447+0000","last_fullsized":"2024-09-15T21:29:45.744447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19481,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746259+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:45.746259+0000","last_peered":"2024-09-15T21:29:45.746259+0000","last_clean":"2024-09-15T21:29:45.746259+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:45.746259+0000","last_undegraded":"2024-09-15T21:29:45.746259+0000","last_fullsized":"2024-09-15T21:29:45.746259+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745428+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:45.745428+0000","last_peered":"2024-09-15T21:29:45.745428+0000","last_clean":"2024-09-15T21:29:45.745428+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:45.745428+0000","last_undegraded":"2024-09-15T21:29:45.745428+0000","last_fullsized":"2024-09-15T21:29:45.745428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25354,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745977+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:45.745977+0000","last_peered":"2024-09-15T21:29:45.745977+0000","last_clean":"2024-09-15T21:29:45.745977+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:45.745977+0000","last_undegraded":"2024-09-15T21:29:45.745977+0000","last_fullsized":"2024-09-15T21:29:45.745977+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745901+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:45.745901+0000","last_peered":"2024-09-15T21:29:45.745901+0000","last_clean":"2024-09-15T21:29:45.745901+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:45.745901+0000","last_undegraded":"2024-09-15T21:29:45.745901+0000","last_fullsized":"2024-09-15T21:29:45.745901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746582+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:45.746582+0000","last_peered":"2024-09-15T21:29:45.746582+0000","last_clean":"2024-09-15T21:29:45.746582+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:45.746582+0000","last_undegraded":"2024-09-15T21:29:45.746582+0000","last_fullsized":"2024-09-15T21:29:45.746582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25740,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745192+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:45.745192+0000","last_peered":"2024-09-15T21:29:45.745192+0000","last_clean":"2024-09-15T21:29:45.745192+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:45.745192+0000","last_undegraded":"2024-09-15T21:29:45.745192+0000","last_fullsized":"2024-09-15T21:29:45.745192+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34476,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96358400,"data_stored":96344484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473700,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898140,"statfs":{"total":96636764160,"available":96151695360,"internally_reserved":0,"allocated":540672,"data_stored":177497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:48.935 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:49.290 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:49.308 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:29:48.640245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34477,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312396,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803124,"statfs":{"total":289910292480,"available":288566398976,"internally_reserved":0,"allocated":97923072,"data_stored":96865843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002209"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25823,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745572+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:45.745572+0000","last_peered":"2024-09-15T21:29:45.745572+0000","last_clean":"2024-09-15T21:29:45.745572+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:45.745572+0000","last_undegraded":"2024-09-15T21:29:45.745572+0000","last_fullsized":"2024-09-15T21:29:45.745572+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19225,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745190+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:45.745190+0000","last_peered":"2024-09-15T21:29:45.745190+0000","last_clean":"2024-09-15T21:29:45.745190+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:45.745190+0000","last_undegraded":"2024-09-15T21:29:45.745190+0000","last_fullsized":"2024-09-15T21:29:45.745190+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19663,"reported_epoch":63,"state":"peering","last_fresh":"2024-09-15T21:29:44.899174+0000","last_change":"2024-09-15T21:29:46.319113+0000","last_active":"2024-09-15T21:29:46.319113+0000","last_peered":"2024-09-15T21:29:44.899174+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:44.899174+0000","last_undegraded":"2024-09-15T21:29:44.899174+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.743935+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:45.743935+0000","last_peered":"2024-09-15T21:29:45.743935+0000","last_clean":"2024-09-15T21:29:45.743935+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:45.743935+0000","last_undegraded":"2024-09-15T21:29:45.743935+0000","last_fullsized":"2024-09-15T21:29:45.743935+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25226,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744775+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:45.744775+0000","last_peered":"2024-09-15T21:29:45.744775+0000","last_clean":"2024-09-15T21:29:45.744775+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:45.744775+0000","last_undegraded":"2024-09-15T21:29:45.744775+0000","last_fullsized":"2024-09-15T21:29:45.744775+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18909,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:45.295640+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:45.295640+0000","last_peered":"2024-09-15T21:29:45.295640+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:45.295640+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2985}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":117,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744447+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:45.744447+0000","last_peered":"2024-09-15T21:29:45.744447+0000","last_clean":"2024-09-15T21:29:45.744447+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:45.744447+0000","last_undegraded":"2024-09-15T21:29:45.744447+0000","last_fullsized":"2024-09-15T21:29:45.744447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19481,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746259+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:45.746259+0000","last_peered":"2024-09-15T21:29:45.746259+0000","last_clean":"2024-09-15T21:29:45.746259+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:45.746259+0000","last_undegraded":"2024-09-15T21:29:45.746259+0000","last_fullsized":"2024-09-15T21:29:45.746259+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745428+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:45.745428+0000","last_peered":"2024-09-15T21:29:45.745428+0000","last_clean":"2024-09-15T21:29:45.745428+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:45.745428+0000","last_undegraded":"2024-09-15T21:29:45.745428+0000","last_fullsized":"2024-09-15T21:29:45.745428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25354,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745977+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:45.745977+0000","last_peered":"2024-09-15T21:29:45.745977+0000","last_clean":"2024-09-15T21:29:45.745977+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:45.745977+0000","last_undegraded":"2024-09-15T21:29:45.745977+0000","last_fullsized":"2024-09-15T21:29:45.745977+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745901+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:45.745901+0000","last_peered":"2024-09-15T21:29:45.745901+0000","last_clean":"2024-09-15T21:29:45.745901+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:45.745901+0000","last_undegraded":"2024-09-15T21:29:45.745901+0000","last_fullsized":"2024-09-15T21:29:45.745901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746582+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:45.746582+0000","last_peered":"2024-09-15T21:29:45.746582+0000","last_clean":"2024-09-15T21:29:45.746582+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:45.746582+0000","last_undegraded":"2024-09-15T21:29:45.746582+0000","last_fullsized":"2024-09-15T21:29:45.746582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25740,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745192+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:45.745192+0000","last_peered":"2024-09-15T21:29:45.745192+0000","last_clean":"2024-09-15T21:29:45.745192+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:45.745192+0000","last_undegraded":"2024-09-15T21:29:45.745192+0000","last_fullsized":"2024-09-15T21:29:45.745192+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34476,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96358400,"data_stored":96344484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473700,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898140,"statfs":{"total":96636764160,"available":96151695360,"internally_reserved":0,"allocated":540672,"data_stored":177497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:49.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:49.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "54'18904", 'reported_seq': 19663, 'reported_epoch': 63, 'state': 'peering', 'last_fresh': '2024-09-15T21:29:44.899174+0000', 'last_change': '2024-09-15T21:29:46.319113+0000', 'last_active': '2024-09-15T21:29:46.319113+0000', 'last_peered': '2024-09-15T21:29:44.899174+0000', 'last_clean': '2024-09-15T21:18:41.345061+0000', 'last_became_active': '2024-09-15T21:18:43.884603+0000', 'last_became_peered': '2024-09-15T21:18:43.884603+0000', 'last_unstale': '2024-09-15T21:29:44.899174+0000', 'last_undegraded': '2024-09-15T21:29:44.899174+0000', 'last_fullsized': '2024-09-15T21:18:43.373039+0000', 'mapping_epoch': 63, 'log_start': "46'8900", 'ondisk_log_start': "46'8900", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2404", 'last_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.588315+0000', 'objects_scrubbed': 2358, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16540, 'num_write_kb': 28364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3473, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:29:49.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:49.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 18909, 'reported_epoch': 62, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:45.295640+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:29:45.295640+0000', 'last_peered': '2024-09-15T21:29:45.295640+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:45.295640+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 358, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2985}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:49.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19063, 'reported_epoch': 62, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.826496+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:14.826496+0000', 'last_peered': '2024-09-15T21:29:14.826496+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:14.826496+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.310 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:49.756 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:49.756 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6089/6 objects degraded (101483.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":802,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1343893504,"bytes_avail":288566398976,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":6089,"degraded_total":6,"degraded_ratio":1014.8333333333335,"recovering_objects_per_sec":77,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:49.776 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:49.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:29:50.101+0000 7f11c1800640 0 --2- 172.21.15.114:0/1826394118 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f11bc121860 0x7f11bc094920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:29:50.336 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:50.337 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:50.355 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:29:48.640245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34477,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312396,"kb_used_data":95628,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281803124,"statfs":{"total":289910292480,"available":288566398976,"internally_reserved":0,"allocated":97923072,"data_stored":96865843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002209"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25366,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826784+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:14.826784+0000","last_peered":"2024-09-15T21:29:14.826784+0000","last_clean":"2024-09-15T21:29:14.826784+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:14.826784+0000","last_undegraded":"2024-09-15T21:29:14.826784+0000","last_fullsized":"2024-09-15T21:29:14.826784+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18772,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823252+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:14.823252+0000","last_peered":"2024-09-15T21:29:14.823252+0000","last_clean":"2024-09-15T21:29:14.823252+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:14.823252+0000","last_undegraded":"2024-09-15T21:29:14.823252+0000","last_fullsized":"2024-09-15T21:29:14.823252+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25823,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745572+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:45.745572+0000","last_peered":"2024-09-15T21:29:45.745572+0000","last_clean":"2024-09-15T21:29:45.745572+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:45.745572+0000","last_undegraded":"2024-09-15T21:29:45.745572+0000","last_fullsized":"2024-09-15T21:29:45.745572+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18675,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823310+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:14.823310+0000","last_peered":"2024-09-15T21:29:14.823310+0000","last_clean":"2024-09-15T21:29:14.823310+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:14.823310+0000","last_undegraded":"2024-09-15T21:29:14.823310+0000","last_fullsized":"2024-09-15T21:29:14.823310+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18877,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826504+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:14.826504+0000","last_peered":"2024-09-15T21:29:14.826504+0000","last_clean":"2024-09-15T21:29:14.826504+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:14.826504+0000","last_undegraded":"2024-09-15T21:29:14.826504+0000","last_fullsized":"2024-09-15T21:29:14.826504+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19225,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745190+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:45.745190+0000","last_peered":"2024-09-15T21:29:45.745190+0000","last_clean":"2024-09-15T21:29:45.745190+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:45.745190+0000","last_undegraded":"2024-09-15T21:29:45.745190+0000","last_fullsized":"2024-09-15T21:29:45.745190+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19663,"reported_epoch":63,"state":"peering","last_fresh":"2024-09-15T21:29:44.899174+0000","last_change":"2024-09-15T21:29:46.319113+0000","last_active":"2024-09-15T21:29:46.319113+0000","last_peered":"2024-09-15T21:29:44.899174+0000","last_clean":"2024-09-15T21:18:41.345061+0000","last_became_active":"2024-09-15T21:18:43.884603+0000","last_became_peered":"2024-09-15T21:18:43.884603+0000","last_unstale":"2024-09-15T21:29:44.899174+0000","last_undegraded":"2024-09-15T21:29:44.899174+0000","last_fullsized":"2024-09-15T21:18:43.373039+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18559,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826153+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:14.826153+0000","last_peered":"2024-09-15T21:29:14.826153+0000","last_clean":"2024-09-15T21:29:14.826153+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:14.826153+0000","last_undegraded":"2024-09-15T21:29:14.826153+0000","last_fullsized":"2024-09-15T21:29:14.826153+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.743935+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:45.743935+0000","last_peered":"2024-09-15T21:29:45.743935+0000","last_clean":"2024-09-15T21:29:45.743935+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:45.743935+0000","last_undegraded":"2024-09-15T21:29:45.743935+0000","last_fullsized":"2024-09-15T21:29:45.743935+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25226,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744775+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:45.744775+0000","last_peered":"2024-09-15T21:29:45.744775+0000","last_clean":"2024-09-15T21:29:45.744775+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:45.744775+0000","last_undegraded":"2024-09-15T21:29:45.744775+0000","last_fullsized":"2024-09-15T21:29:45.744775+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823072+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:14.823072+0000","last_peered":"2024-09-15T21:29:14.823072+0000","last_clean":"2024-09-15T21:29:14.823072+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:14.823072+0000","last_undegraded":"2024-09-15T21:29:14.823072+0000","last_fullsized":"2024-09-15T21:29:14.823072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":18909,"reported_epoch":62,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:45.295640+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:45.295640+0000","last_peered":"2024-09-15T21:29:45.295640+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:45.295640+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2985}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":136,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.826227+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:14.826227+0000","last_peered":"2024-09-15T21:29:14.826227+0000","last_clean":"2024-09-15T21:29:14.826227+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:14.826227+0000","last_undegraded":"2024-09-15T21:29:14.826227+0000","last_fullsized":"2024-09-15T21:29:14.826227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24972,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823212+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:14.823212+0000","last_peered":"2024-09-15T21:29:14.823212+0000","last_clean":"2024-09-15T21:29:14.823212+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:14.823212+0000","last_undegraded":"2024-09-15T21:29:14.823212+0000","last_fullsized":"2024-09-15T21:29:14.823212+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":117,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.744447+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:45.744447+0000","last_peered":"2024-09-15T21:29:45.744447+0000","last_clean":"2024-09-15T21:29:45.744447+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:45.744447+0000","last_undegraded":"2024-09-15T21:29:45.744447+0000","last_fullsized":"2024-09-15T21:29:45.744447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25411,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823142+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:14.823142+0000","last_peered":"2024-09-15T21:29:14.823142+0000","last_clean":"2024-09-15T21:29:14.823142+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:14.823142+0000","last_undegraded":"2024-09-15T21:29:14.823142+0000","last_fullsized":"2024-09-15T21:29:14.823142+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19481,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746259+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:45.746259+0000","last_peered":"2024-09-15T21:29:45.746259+0000","last_clean":"2024-09-15T21:29:45.746259+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:45.746259+0000","last_undegraded":"2024-09-15T21:29:45.746259+0000","last_fullsized":"2024-09-15T21:29:45.746259+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745428+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:45.745428+0000","last_peered":"2024-09-15T21:29:45.745428+0000","last_clean":"2024-09-15T21:29:45.745428+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:45.745428+0000","last_undegraded":"2024-09-15T21:29:45.745428+0000","last_fullsized":"2024-09-15T21:29:45.745428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25354,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745977+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:45.745977+0000","last_peered":"2024-09-15T21:29:45.745977+0000","last_clean":"2024-09-15T21:29:45.745977+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:45.745977+0000","last_undegraded":"2024-09-15T21:29:45.745977+0000","last_fullsized":"2024-09-15T21:29:45.745977+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.823417+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:14.823417+0000","last_peered":"2024-09-15T21:29:14.823417+0000","last_clean":"2024-09-15T21:29:14.823417+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:14.823417+0000","last_undegraded":"2024-09-15T21:29:14.823417+0000","last_fullsized":"2024-09-15T21:29:14.823417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19063,"reported_epoch":62,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.826496+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:14.826496+0000","last_peered":"2024-09-15T21:29:14.826496+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:14.826496+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745901+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:45.745901+0000","last_peered":"2024-09-15T21:29:45.745901+0000","last_clean":"2024-09-15T21:29:45.745901+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:45.745901+0000","last_undegraded":"2024-09-15T21:29:45.745901+0000","last_fullsized":"2024-09-15T21:29:45.745901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.746582+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:45.746582+0000","last_peered":"2024-09-15T21:29:45.746582+0000","last_clean":"2024-09-15T21:29:45.746582+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:45.746582+0000","last_undegraded":"2024-09-15T21:29:45.746582+0000","last_fullsized":"2024-09-15T21:29:45.746582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25740,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.745192+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:45.745192+0000","last_peered":"2024-09-15T21:29:45.745192+0000","last_clean":"2024-09-15T21:29:45.745192+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:45.745192+0000","last_undegraded":"2024-09-15T21:29:45.745192+0000","last_fullsized":"2024-09-15T21:29:45.745192+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34476,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96358400,"data_stored":96344484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953625,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347784,"kb_used_data":41204,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94024056,"statfs":{"total":96636764160,"available":96280633344,"internally_reserved":0,"allocated":42192896,"data_stored":41849647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149850,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490912,"kb_used_data":53896,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880928,"statfs":{"total":96636764160,"available":96134070272,"internally_reserved":0,"allocated":55189504,"data_stored":54838699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473700,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898140,"statfs":{"total":96636764160,"available":96151695360,"internally_reserved":0,"allocated":540672,"data_stored":177497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41676800,"data_stored":41676800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:53.358 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:53.691 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:53.711 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-15T21:29:52.641560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35140,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002324"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744336+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744336+0000","last_peered":"2024-09-15T21:29:46.744336+0000","last_clean":"2024-09-15T21:29:46.744336+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744336+0000","last_undegraded":"2024-09-15T21:29:46.744336+0000","last_fullsized":"2024-09-15T21:29:46.744336+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":20239,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:50.300724+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:50.300724+0000","last_peered":"2024-09-15T21:29:50.300724+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:50.300724+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1021,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748498+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748498+0000","last_peered":"2024-09-15T21:29:46.748498+0000","last_clean":"2024-09-15T21:29:46.748498+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748498+0000","last_undegraded":"2024-09-15T21:29:46.748498+0000","last_fullsized":"2024-09-15T21:29:46.748498+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35139,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:53.712 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:54.041 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:54.061 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-15T21:29:52.641560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35140,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002324"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744336+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744336+0000","last_peered":"2024-09-15T21:29:46.744336+0000","last_clean":"2024-09-15T21:29:46.744336+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744336+0000","last_undegraded":"2024-09-15T21:29:46.744336+0000","last_fullsized":"2024-09-15T21:29:46.744336+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":20239,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:50.300724+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:50.300724+0000","last_peered":"2024-09-15T21:29:50.300724+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:50.300724+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1021,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748498+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748498+0000","last_peered":"2024-09-15T21:29:46.748498+0000","last_clean":"2024-09-15T21:29:46.748498+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748498+0000","last_undegraded":"2024-09-15T21:29:46.748498+0000","last_fullsized":"2024-09-15T21:29:46.748498+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35139,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:54.062 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:54.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 20239, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:50.300724+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:29:50.300724+0000', 'last_peered': '2024-09-15T21:29:50.300724+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:50.300724+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2322, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1021, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2322}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:54.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19067, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.748977+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.748977+0000', 'last_peered': '2024-09-15T21:29:46.748977+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.748977+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:54.474 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5426/6 objects degraded (90433.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":807,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1321205760,"bytes_avail":288589086720,"bytes_total":289910292480,"degraded_objects":5426,"degraded_total":6,"degraded_ratio":904.33333333333348,"recovering_objects_per_sec":160,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:54.495 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:54.495 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:54.827 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:54.827 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:54.845 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-15T21:29:54.642172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35140,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-663,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":663,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.898647"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744336+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744336+0000","last_peered":"2024-09-15T21:29:46.744336+0000","last_clean":"2024-09-15T21:29:46.744336+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744336+0000","last_undegraded":"2024-09-15T21:29:46.744336+0000","last_fullsized":"2024-09-15T21:29:46.744336+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":20239,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:50.300724+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:50.300724+0000","last_peered":"2024-09-15T21:29:50.300724+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:50.300724+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1021,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2322}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748498+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748498+0000","last_peered":"2024-09-15T21:29:46.748498+0000","last_clean":"2024-09-15T21:29:46.748498+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748498+0000","last_undegraded":"2024-09-15T21:29:46.748498+0000","last_fullsized":"2024-09-15T21:29:46.748498+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35139,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953626,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:57.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:58.213 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:58.233 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-15T21:29:56.642520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35543,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002909"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21046,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:55.299805+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:55.299805+0000","last_peered":"2024-09-15T21:29:55.299805+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:55.299805+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1424,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35542,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:58.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:58.556 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:58.574 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-15T21:29:56.642520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35543,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002909"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21046,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:55.299805+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:55.299805+0000","last_peered":"2024-09-15T21:29:55.299805+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:55.299805+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1424,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35542,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:58.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:58.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 21046, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:55.299805+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:29:55.299805+0000', 'last_peered': '2024-09-15T21:29:55.299805+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:29:55.299805+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1919, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1424, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1919}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:58.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:29:58.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19067, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.748977+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.748977+0000', 'last_peered': '2024-09-15T21:29:46.748977+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.748977+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:58.576 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:58.988 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:58.988 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5023/6 objects degraded (83716.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":812,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1325465600,"bytes_avail":288584826880,"bytes_total":289910292480,"degraded_objects":5023,"degraded_total":6,"degraded_ratio":837.16666666666652,"recovering_objects_per_sec":89,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:59.008 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:59.008 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:29:59.322 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:29:59.340 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-15T21:29:58.643184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35543,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1066,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.899931"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21046,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:55.299805+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:29:55.299805+0000","last_peered":"2024-09-15T21:29:55.299805+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:29:55.299805+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1424,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35542,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953627,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:02.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:02.679 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:02.679 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:02.699 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-15T21:30:02.644310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35965,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":825,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:00.303277+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:00.303277+0000","last_peered":"2024-09-15T21:30:00.303277+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:00.303277+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35964,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953628,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:02.700 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:03.044 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:03.044 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:03.066 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-15T21:30:02.644310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35965,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":825,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:00.303277+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:00.303277+0000","last_peered":"2024-09-15T21:30:00.303277+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:00.303277+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35964,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953628,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:03.067 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:03.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 21890, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:00.303277+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:30:00.303277+0000', 'last_peered': '2024-09-15T21:30:00.303277+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:30:00.303277+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1497, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1846, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1497}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:03.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:03.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19067, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.748977+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.748977+0000', 'last_peered': '2024-09-15T21:29:46.748977+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.748977+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:03.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:03.478 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4601/6 objects degraded (76683.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":816,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1325465600,"bytes_avail":288584826880,"bytes_total":289910292480,"degraded_objects":4601,"degraded_total":6,"degraded_ratio":766.83333333333348,"recovering_objects_per_sec":68,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":825,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-15T21:28:06.609765+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:03.499 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:03.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:03.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:30:03.821+0000 7fd7e0af4640 0 --2- 172.21.15.114:0/2950533698 >> [v2:172.21.15.114:3302/0,v1:172.21.15.114:6791/0] conn(0x7fd7dc12c2e0 0x7fd7dc1bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:30:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:04.013 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:04.030 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-15T21:30:02.644310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35965,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294400,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821120,"statfs":{"total":289910292480,"available":288584826880,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":825,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":21890,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:00.303277+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:00.303277+0000","last_peered":"2024-09-15T21:30:00.303277+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:00.303277+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19067,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.748977+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.748977+0000","last_peered":"2024-09-15T21:29:46.748977+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.748977+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4601,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35964,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953628,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626572,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469544,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902296,"statfs":{"total":96636764160,"available":96155951104,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:07.032 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:07.361 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:07.379 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:30:06.645386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36483,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1298560,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281816960,"statfs":{"total":289910292480,"available":288580567040,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003214"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":22926,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:05.307052+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:05.307052+0000","last_peered":"2024-09-15T21:30:05.307052+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:05.307052+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-979}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36482,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473704,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898136,"statfs":{"total":96636764160,"available":96151691264,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:07.381 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:07.697 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:07.716 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:30:06.645386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36483,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1298560,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281816960,"statfs":{"total":289910292480,"available":288580567040,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003214"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":22926,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:05.307052+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:05.307052+0000","last_peered":"2024-09-15T21:30:05.307052+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:05.307052+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-979}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36482,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473704,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898136,"statfs":{"total":96636764160,"available":96151691264,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:07.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:07.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 22926, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:05.307052+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:30:05.307052+0000', 'last_peered': '2024-09-15T21:30:05.307052+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:30:05.307052+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2364, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -979}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:07.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:07.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19068, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.750759+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.750759+0000', 'last_peered': '2024-09-15T21:29:46.750759+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.750759+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:07.718 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:08.111 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4083/6 objects degraded (68050.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":821,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1329725440,"bytes_avail":288580567040,"bytes_total":289910292480,"degraded_objects":4083,"degraded_total":6,"degraded_ratio":680.5,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1343,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:08.131 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:08.131 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:08.470 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:30:06.645386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36483,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1298560,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281816960,"statfs":{"total":289910292480,"available":288580567040,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1343,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003214"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743881+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743881+0000","last_peered":"2024-09-15T21:29:46.743881+0000","last_clean":"2024-09-15T21:29:46.743881+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743881+0000","last_undegraded":"2024-09-15T21:29:46.743881+0000","last_fullsized":"2024-09-15T21:29:46.743881+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":22926,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:05.307052+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:05.307052+0000","last_peered":"2024-09-15T21:30:05.307052+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:05.307052+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-979}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24976,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744205+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744205+0000","last_peered":"2024-09-15T21:29:46.744205+0000","last_clean":"2024-09-15T21:29:46.744205+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744205+0000","last_undegraded":"2024-09-15T21:29:46.744205+0000","last_fullsized":"2024-09-15T21:29:46.744205+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744914+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.744914+0000","last_peered":"2024-09-15T21:29:46.744914+0000","last_clean":"2024-09-15T21:29:46.744914+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.744914+0000","last_undegraded":"2024-09-15T21:29:46.744914+0000","last_fullsized":"2024-09-15T21:29:46.744914+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36482,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953629,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473704,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898136,"statfs":{"total":96636764160,"available":96151691264,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:11.473 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:11.797 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:11.797 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:11.816 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-15T21:30:10.646562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1298560,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281816960,"statfs":{"total":289910292480,"available":288580567040,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743990+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743990+0000","last_peered":"2024-09-15T21:29:46.743990+0000","last_clean":"2024-09-15T21:29:46.743990+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743990+0000","last_undegraded":"2024-09-15T21:29:46.743990+0000","last_fullsized":"2024-09-15T21:29:46.743990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24098,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:10.308649+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:10.308649+0000","last_peered":"2024-09-15T21:30:10.308649+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:10.308649+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744350+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744350+0000","last_peered":"2024-09-15T21:29:46.744350+0000","last_clean":"2024-09-15T21:29:46.744350+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744350+0000","last_undegraded":"2024-09-15T21:29:46.744350+0000","last_fullsized":"2024-09-15T21:29:46.744350+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745019+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.745019+0000","last_peered":"2024-09-15T21:29:46.745019+0000","last_clean":"2024-09-15T21:29:46.745019+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.745019+0000","last_undegraded":"2024-09-15T21:29:46.745019+0000","last_fullsized":"2024-09-15T21:29:46.745019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473704,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898136,"statfs":{"total":96636764160,"available":96151691264,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:11.818 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:12.115 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:12.115 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:12.134 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-15T21:30:10.646562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1298560,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281816960,"statfs":{"total":289910292480,"available":288580567040,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743990+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743990+0000","last_peered":"2024-09-15T21:29:46.743990+0000","last_clean":"2024-09-15T21:29:46.743990+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743990+0000","last_undegraded":"2024-09-15T21:29:46.743990+0000","last_fullsized":"2024-09-15T21:29:46.743990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24098,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:10.308649+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:10.308649+0000","last_peered":"2024-09-15T21:30:10.308649+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:10.308649+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744350+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744350+0000","last_peered":"2024-09-15T21:29:46.744350+0000","last_clean":"2024-09-15T21:29:46.744350+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744350+0000","last_undegraded":"2024-09-15T21:29:46.744350+0000","last_fullsized":"2024-09-15T21:29:46.744350+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745019+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.745019+0000","last_peered":"2024-09-15T21:29:46.745019+0000","last_clean":"2024-09-15T21:29:46.745019+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.745019+0000","last_undegraded":"2024-09-15T21:29:46.745019+0000","last_fullsized":"2024-09-15T21:29:46.745019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473704,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898136,"statfs":{"total":96636764160,"available":96151691264,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:12.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:12.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 24098, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:10.308649+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:30:10.308649+0000', 'last_peered': '2024-09-15T21:30:10.308649+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:30:10.308649+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 393, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2950, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -393}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:12.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:12.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19068, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.750759+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.750759+0000', 'last_peered': '2024-09-15T21:29:46.750759+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.750759+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:12.136 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:12.535 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3497/6 objects degraded (58283.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":825,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1329725440,"bytes_avail":288580567040,"bytes_total":289910292480,"degraded_objects":3497,"degraded_total":6,"degraded_ratio":582.83333333333337,"recovering_objects_per_sec":127,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1526,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:12.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:12.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:12.872 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:12.891 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-15T21:30:12.647123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003499"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743990+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743990+0000","last_peered":"2024-09-15T21:29:46.743990+0000","last_clean":"2024-09-15T21:29:46.743990+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743990+0000","last_undegraded":"2024-09-15T21:29:46.743990+0000","last_fullsized":"2024-09-15T21:29:46.743990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24098,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:10.308649+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:10.308649+0000","last_peered":"2024-09-15T21:30:10.308649+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:10.308649+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744350+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744350+0000","last_peered":"2024-09-15T21:29:46.744350+0000","last_clean":"2024-09-15T21:29:46.744350+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744350+0000","last_undegraded":"2024-09-15T21:29:46.744350+0000","last_fullsized":"2024-09-15T21:29:46.744350+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745019+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.745019+0000","last_peered":"2024-09-15T21:29:46.745019+0000","last_clean":"2024-09-15T21:29:46.745019+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.745019+0000","last_undegraded":"2024-09-15T21:29:46.745019+0000","last_fullsized":"2024-09-15T21:29:46.745019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:15.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:16.255 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:16.256 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:16.280 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-15T21:30:14.730247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":586,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.084860"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743990+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743990+0000","last_peered":"2024-09-15T21:29:46.743990+0000","last_clean":"2024-09-15T21:29:46.743990+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743990+0000","last_undegraded":"2024-09-15T21:29:46.743990+0000","last_fullsized":"2024-09-15T21:29:46.743990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24098,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:10.308649+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:10.308649+0000","last_peered":"2024-09-15T21:30:10.308649+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:10.308649+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744350+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744350+0000","last_peered":"2024-09-15T21:29:46.744350+0000","last_clean":"2024-09-15T21:29:46.744350+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744350+0000","last_undegraded":"2024-09-15T21:29:46.744350+0000","last_fullsized":"2024-09-15T21:29:46.744350+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745019+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.745019+0000","last_peered":"2024-09-15T21:29:46.745019+0000","last_clean":"2024-09-15T21:29:46.745019+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.745019+0000","last_undegraded":"2024-09-15T21:29:46.745019+0000","last_fullsized":"2024-09-15T21:29:46.745019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:16.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:16.617 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:16.617 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:16.638 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-15T21:30:14.730247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37069,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1290240,"kb_used_data":81792,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281825280,"statfs":{"total":289910292480,"available":288589086720,"internally_reserved":0,"allocated":83755008,"data_stored":82670656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-586,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":586,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.084860"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25370,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.749295+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:29:46.749295+0000","last_peered":"2024-09-15T21:29:46.749295+0000","last_clean":"2024-09-15T21:29:46.749295+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:29:46.749295+0000","last_undegraded":"2024-09-15T21:29:46.749295+0000","last_fullsized":"2024-09-15T21:29:46.749295+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744442+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:29:46.744442+0000","last_peered":"2024-09-15T21:29:46.744442+0000","last_clean":"2024-09-15T21:29:46.744442+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:29:46.744442+0000","last_undegraded":"2024-09-15T21:29:46.744442+0000","last_fullsized":"2024-09-15T21:29:46.744442+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744487+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:29:46.744487+0000","last_peered":"2024-09-15T21:29:46.744487+0000","last_clean":"2024-09-15T21:29:46.744487+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:29:46.744487+0000","last_undegraded":"2024-09-15T21:29:46.744487+0000","last_fullsized":"2024-09-15T21:29:46.744487+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18679,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744372+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:29:46.744372+0000","last_peered":"2024-09-15T21:29:46.744372+0000","last_clean":"2024-09-15T21:29:46.744372+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:29:46.744372+0000","last_undegraded":"2024-09-15T21:29:46.744372+0000","last_fullsized":"2024-09-15T21:29:46.744372+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748792+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:29:46.748792+0000","last_peered":"2024-09-15T21:29:46.748792+0000","last_clean":"2024-09-15T21:29:46.748792+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:29:46.748792+0000","last_undegraded":"2024-09-15T21:29:46.748792+0000","last_fullsized":"2024-09-15T21:29:46.748792+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19227,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743999+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:29:46.743999+0000","last_peered":"2024-09-15T21:29:46.743999+0000","last_clean":"2024-09-15T21:29:46.743999+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:29:46.743999+0000","last_undegraded":"2024-09-15T21:29:46.743999+0000","last_fullsized":"2024-09-15T21:29:46.743999+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19672,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.844527+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:29:46.844527+0000","last_peered":"2024-09-15T21:29:46.844527+0000","last_clean":"2024-09-15T21:29:46.844527+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:29:46.844527+0000","last_undegraded":"2024-09-15T21:29:46.844527+0000","last_fullsized":"2024-09-15T21:29:46.844527+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18563,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748261+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:29:46.748261+0000","last_peered":"2024-09-15T21:29:46.748261+0000","last_clean":"2024-09-15T21:29:46.748261+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:29:46.748261+0000","last_undegraded":"2024-09-15T21:29:46.748261+0000","last_fullsized":"2024-09-15T21:29:46.748261+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743351+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:29:46.743351+0000","last_peered":"2024-09-15T21:29:46.743351+0000","last_clean":"2024-09-15T21:29:46.743351+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:29:46.743351+0000","last_undegraded":"2024-09-15T21:29:46.743351+0000","last_fullsized":"2024-09-15T21:29:46.743351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25229,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743528+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:29:46.743528+0000","last_peered":"2024-09-15T21:29:46.743528+0000","last_clean":"2024-09-15T21:29:46.743528+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:29:46.743528+0000","last_undegraded":"2024-09-15T21:29:46.743528+0000","last_fullsized":"2024-09-15T21:29:46.743528+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743990+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:29:46.743990+0000","last_peered":"2024-09-15T21:29:46.743990+0000","last_clean":"2024-09-15T21:29:46.743990+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:29:46.743990+0000","last_undegraded":"2024-09-15T21:29:46.743990+0000","last_fullsized":"2024-09-15T21:29:46.743990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24098,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:10.308649+0000","last_change":"2024-09-15T21:29:44.899639+0000","last_active":"2024-09-15T21:30:10.308649+0000","last_peered":"2024-09-15T21:30:10.308649+0000","last_clean":"2024-09-15T21:18:41.348908+0000","last_became_active":"2024-09-15T21:18:43.884398+0000","last_became_peered":"2024-09-15T21:18:43.884398+0000","last_unstale":"2024-09-15T21:30:10.308649+0000","last_undegraded":"2024-09-15T21:18:43.480326+0000","last_fullsized":"2024-09-15T21:18:43.370239+0000","mapping_epoch":43,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2950,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":141,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.748576+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:29:46.748576+0000","last_peered":"2024-09-15T21:29:46.748576+0000","last_clean":"2024-09-15T21:29:46.748576+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:29:46.748576+0000","last_undegraded":"2024-09-15T21:29:46.748576+0000","last_fullsized":"2024-09-15T21:29:46.748576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744350+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:29:46.744350+0000","last_peered":"2024-09-15T21:29:46.744350+0000","last_clean":"2024-09-15T21:29:46.744350+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:29:46.744350+0000","last_undegraded":"2024-09-15T21:29:46.744350+0000","last_fullsized":"2024-09-15T21:29:46.744350+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":119,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743122+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:29:46.743122+0000","last_peered":"2024-09-15T21:29:46.743122+0000","last_clean":"2024-09-15T21:29:46.743122+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:29:46.743122+0000","last_undegraded":"2024-09-15T21:29:46.743122+0000","last_fullsized":"2024-09-15T21:29:46.743122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25415,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743459+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:29:46.743459+0000","last_peered":"2024-09-15T21:29:46.743459+0000","last_clean":"2024-09-15T21:29:46.743459+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:29:46.743459+0000","last_undegraded":"2024-09-15T21:29:46.743459+0000","last_fullsized":"2024-09-15T21:29:46.743459+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19483,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745646+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:29:46.745646+0000","last_peered":"2024-09-15T21:29:46.745646+0000","last_clean":"2024-09-15T21:29:46.745646+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:29:46.745646+0000","last_undegraded":"2024-09-15T21:29:46.745646+0000","last_fullsized":"2024-09-15T21:29:46.745646+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743952+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:29:46.743952+0000","last_peered":"2024-09-15T21:29:46.743952+0000","last_clean":"2024-09-15T21:29:46.743952+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:29:46.743952+0000","last_undegraded":"2024-09-15T21:29:46.743952+0000","last_fullsized":"2024-09-15T21:29:46.743952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745110+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:29:46.745110+0000","last_peered":"2024-09-15T21:29:46.745110+0000","last_clean":"2024-09-15T21:29:46.745110+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:29:46.745110+0000","last_undegraded":"2024-09-15T21:29:46.745110+0000","last_fullsized":"2024-09-15T21:29:46.745110+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745019+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:29:46.745019+0000","last_peered":"2024-09-15T21:29:46.745019+0000","last_clean":"2024-09-15T21:29:46.745019+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:29:46.745019+0000","last_undegraded":"2024-09-15T21:29:46.745019+0000","last_fullsized":"2024-09-15T21:29:46.745019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19068,"reported_epoch":64,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:46.750759+0000","last_change":"2024-09-15T21:18:43.909295+0000","last_active":"2024-09-15T21:29:46.750759+0000","last_peered":"2024-09-15T21:29:46.750759+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:29:46.750759+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.744864+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:29:46.744864+0000","last_peered":"2024-09-15T21:29:46.744864+0000","last_clean":"2024-09-15T21:29:46.744864+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:29:46.744864+0000","last_undegraded":"2024-09-15T21:29:46.744864+0000","last_fullsized":"2024-09-15T21:29:46.744864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.745406+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:29:46.745406+0000","last_peered":"2024-09-15T21:29:46.745406+0000","last_clean":"2024-09-15T21:29:46.745406+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:29:46.745406+0000","last_undegraded":"2024-09-15T21:29:46.745406+0000","last_fullsized":"2024-09-15T21:29:46.745406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25742,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-15T21:29:46.743694+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:29:46.743694+0000","last_peered":"2024-09-15T21:29:46.743694+0000","last_clean":"2024-09-15T21:29:46.743694+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:29:46.743694+0000","last_undegraded":"2024-09-15T21:29:46.743694+0000","last_fullsized":"2024-09-15T21:29:46.743694+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37068,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":82149376,"data_stored":82135460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953630,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333928,"kb_used_data":27344,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037912,"statfs":{"total":96636764160,"available":96294821888,"internally_reserved":0,"allocated":28000256,"data_stored":27646162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":53912,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":55205888,"data_stored":54844238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626574,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465384,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906456,"statfs":{"total":96636764160,"available":96160210944,"internally_reserved":0,"allocated":548864,"data_stored":180256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27467776,"data_stored":27467776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:16.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:16.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "54'18034", 'reported_seq': 24098, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:10.308649+0000', 'last_change': '2024-09-15T21:29:44.899639+0000', 'last_active': '2024-09-15T21:30:10.308649+0000', 'last_peered': '2024-09-15T21:30:10.308649+0000', 'last_clean': '2024-09-15T21:18:41.348908+0000', 'last_became_active': '2024-09-15T21:18:43.884398+0000', 'last_became_peered': '2024-09-15T21:18:43.884398+0000', 'last_unstale': '2024-09-15T21:30:10.308649+0000', 'last_undegraded': '2024-09-15T21:18:43.480326+0000', 'last_fullsized': '2024-09-15T21:18:43.370239+0000', 'mapping_epoch': 43, 'log_start': "46'8000", 'ondisk_log_start': "46'8000", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2513", 'last_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:24.693269+0000', 'objects_scrubbed': 2466, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 971, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 393, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15560, 'num_write_kb': 26188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2950, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -393}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:16.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19068, 'reported_epoch': 64, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:46.750759+0000', 'last_change': '2024-09-15T21:18:43.909295+0000', 'last_active': '2024-09-15T21:29:46.750759+0000', 'last_peered': '2024-09-15T21:29:46.750759+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:29:46.750759+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.640 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:17.072 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:17.072 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2824/6 objects degraded (47066.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":830,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1325481984,"bytes_avail":288584810496,"bytes_total":289910292480,"degraded_objects":2824,"degraded_total":6,"degraded_ratio":470.66666666666674,"recovering_objects_per_sec":157,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1259,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:17.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:17.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:17.495 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:17.496 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:17.515 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-15T21:30:16.648025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37742,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294416,"kb_used_data":81816,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281821104,"statfs":{"total":289910292480,"available":288584810496,"internally_reserved":0,"allocated":83779584,"data_stored":82680421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001980"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19633,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:15.317687+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:15.317687+0000","last_peered":"2024-09-15T21:30:15.317687+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:15.317687+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":543,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37741,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953631,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":333932,"kb_used_data":27352,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94037908,"statfs":{"total":96636764160,"available":96294817792,"internally_reserved":0,"allocated":28008448,"data_stored":27648917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490932,"kb_used_data":53920,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880908,"statfs":{"total":96636764160,"available":96134049792,"internally_reserved":0,"allocated":55214080,"data_stored":54846993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:20.518 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:20.858 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:20.859 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:20.878 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-15T21:30:20.649041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001917"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:20.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:21.204 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:21.205 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:21.223 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-15T21:30:20.649041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001917"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:21.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:21.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 21141, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:20.314008+0000', 'last_change': '2024-09-15T21:30:12.884321+0000', 'last_active': '2024-09-15T21:30:20.314008+0000', 'last_peered': '2024-09-15T21:30:20.314008+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:20.314008+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1297, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:21.225 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:21.685 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:21.685 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2070/6 objects degraded (34500.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":834,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1311817728,"bytes_avail":288598474752,"bytes_total":289910292480,"degraded_objects":2070,"degraded_total":6,"degraded_ratio":345,"recovering_objects_per_sec":178,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:21.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:21.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:22.035 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:22.036 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:22.057 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-15T21:30:20.649041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001917"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:25.060 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:25.400 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:25.401 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:25.420 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-15T21:30:24.650276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285232,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281830288,"statfs":{"total":289910292480,"available":288594214912,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002664"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473712,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898128,"statfs":{"total":96636764160,"available":96151683072,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:25.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:25.730 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:25.731 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:25.750 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-15T21:30:24.650276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285232,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281830288,"statfs":{"total":289910292480,"available":288594214912,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002664"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473712,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898128,"statfs":{"total":96636764160,"available":96151683072,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:25.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:25.751 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 21141, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:20.314008+0000', 'last_change': '2024-09-15T21:30:12.884321+0000', 'last_active': '2024-09-15T21:30:20.314008+0000', 'last_peered': '2024-09-15T21:30:20.314008+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:20.314008+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1297, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:25.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:26.130 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:26.130 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2070/6 objects degraded (34500.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":839,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1316077568,"bytes_avail":288594214912,"bytes_total":289910292480,"degraded_objects":2070,"degraded_total":6,"degraded_ratio":345,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:26.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:26.152 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:26.462 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:26.462 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:26.481 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-15T21:30:24.650276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38496,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285232,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281830288,"statfs":{"total":289910292480,"available":288594214912,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002664"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18567,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734887+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734887+0000","last_peered":"2024-09-15T21:30:14.734887+0000","last_clean":"2024-09-15T21:30:14.734887+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734887+0000","last_undegraded":"2024-09-15T21:30:14.734887+0000","last_fullsized":"2024-09-15T21:30:14.734887+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21141,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:20.314008+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:20.314008+0000","last_peered":"2024-09-15T21:30:20.314008+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:20.314008+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25746,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731799+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.731799+0000","last_peered":"2024-09-15T21:30:14.731799+0000","last_clean":"2024-09-15T21:30:14.731799+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.731799+0000","last_undegraded":"2024-09-15T21:30:14.731799+0000","last_fullsized":"2024-09-15T21:30:14.731799+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38495,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953632,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473712,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898128,"statfs":{"total":96636764160,"available":96151683072,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:29.483 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:29.834 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:29.834 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:29.853 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-15T21:30:28.651564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38916,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1272752,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281842768,"statfs":{"total":289910292480,"available":288606994432,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003538"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21981,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:25.316789+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:25.316789+0000","last_peered":"2024-09-15T21:30:25.316789+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:25.316789+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38915,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953633,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461232,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910608,"statfs":{"total":96636764160,"available":96164462592,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:29.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:30.182 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:30.183 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:30.202 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-15T21:30:28.651564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38916,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1272752,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281842768,"statfs":{"total":289910292480,"available":288606994432,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003538"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":21981,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:25.316789+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:25.316789+0000","last_peered":"2024-09-15T21:30:25.316789+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:25.316789+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38915,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953633,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149858,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461232,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910608,"statfs":{"total":96636764160,"available":96164462592,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:30.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:30.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 21981, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:25.316789+0000', 'last_change': '2024-09-15T21:30:12.884321+0000', 'last_active': '2024-09-15T21:30:25.316789+0000', 'last_peered': '2024-09-15T21:30:25.316789+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:25.316789+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1650, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1717, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1650}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:30.204 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:30.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-15T21:30:30.537+0000 7f1bd659c640 0 --2- 172.21.15.114:0/118185695 >> [v2:172.21.15.114:3301/0,v1:172.21.15.114:6790/0] conn(0x7f1bd8073980 0x7f1bd8129050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:30:30.787 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:30.787 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1650/6 objects degraded (27500.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":843,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1303298048,"bytes_avail":288606994432,"bytes_total":289910292480,"degraded_objects":1650,"degraded_total":6,"degraded_ratio":275,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1174,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:30.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:30.807 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:31.142 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:31.143 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:31.160 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-15T21:30:30.652089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39412,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1272752,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1204211,"kb_avail":281842768,"statfs":{"total":289910292480,"available":288606994432,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1233112074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1670,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003581"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":22973,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.319441+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:30.319441+0000","last_peered":"2024-09-15T21:30:30.319441+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:30.319441+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39411,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626577,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461232,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910608,"statfs":{"total":96636764160,"available":96164462592,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":471723692},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:34.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:34.511 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:34.512 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:34.531 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-15T21:30:32.652658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39412,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1276912,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281838608,"statfs":{"total":289910292480,"available":288602734592,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003617"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":22973,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.319441+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:30.319441+0000","last_peered":"2024-09-15T21:30:30.319441+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:30.319441+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39411,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465392,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906448,"statfs":{"total":96636764160,"available":96160202752,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:34.533 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:34.851 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:34.918 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:34.919 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-15T21:30:34.653241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39412,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1276912,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281838608,"statfs":{"total":289910292480,"available":288602734592,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003576"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":22973,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.319441+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:30.319441+0000","last_peered":"2024-09-15T21:30:30.319441+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:30.319441+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39411,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465392,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906448,"statfs":{"total":96636764160,"available":96160202752,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:34.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:34.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 22973, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.319441+0000', 'last_change': '2024-09-15T21:30:12.884321+0000', 'last_active': '2024-09-15T21:30:30.319441+0000', 'last_peered': '2024-09-15T21:30:30.319441+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:30.319441+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2213, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:34.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:35.305 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1154/6 objects degraded (19233.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":848,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1307557888,"bytes_avail":288602734592,"bytes_total":289910292480,"degraded_objects":1154,"degraded_total":6,"degraded_ratio":192.33333333333337,"recovering_objects_per_sec":76,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":916,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:35.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:35.325 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:35.630 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:35.631 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:35.648 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-15T21:30:34.653241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39412,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1276912,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1208371,"kb_avail":281838608,"statfs":{"total":289910292480,"available":288602734592,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1237371914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003576"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":22973,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.319441+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:30.319441+0000","last_peered":"2024-09-15T21:30:30.319441+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:30.319441+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39411,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953634,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149859,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626578,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465392,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906448,"statfs":{"total":96636764160,"available":96160202752,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":475983532},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:38.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:38.970 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:38.971 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:38.989 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-15T21:30:38.654278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39983,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1067,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":24115,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:35.320259+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:35.320259+0000","last_peered":"2024-09-15T21:30:35.320259+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:35.320259+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39982,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:38.991 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:39.319 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:39.319 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:39.340 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-15T21:30:38.654278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39983,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1067,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":24115,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:35.320259+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:35.320259+0000","last_peered":"2024-09-15T21:30:35.320259+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:35.320259+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39982,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:39.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:39.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 24115, 'reported_epoch': 66, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:35.320259+0000', 'last_change': '2024-09-15T21:30:12.884321+0000', 'last_active': '2024-09-15T21:30:35.320259+0000', 'last_peered': '2024-09-15T21:30:35.320259+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:35.320259+0000', 'last_undegraded': '2024-09-15T21:18:43.487554+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 43, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 583, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2784, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:39.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:39.745 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:39.745 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 583/6 objects degraded (9716.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":852,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1311817728,"bytes_avail":288598474752,"bytes_total":289910292480,"degraded_objects":583,"degraded_total":6,"degraded_ratio":97.166666666666686,"recovering_objects_per_sec":88,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1067,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:39.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:39.766 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:40.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:40.079 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:40.100 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-15T21:30:38.654278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39983,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1281072,"kb_used_data":68468,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281834448,"statfs":{"total":289910292480,"available":288598474752,"internally_reserved":0,"allocated":70111232,"data_stored":69001153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1241631754},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1067,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25830,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731736+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:14.731736+0000","last_peered":"2024-09-15T21:30:14.731736+0000","last_clean":"2024-09-15T21:30:14.731736+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:14.731736+0000","last_undegraded":"2024-09-15T21:30:14.731736+0000","last_fullsized":"2024-09-15T21:30:14.731736+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731465+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:14.731465+0000","last_peered":"2024-09-15T21:30:14.731465+0000","last_clean":"2024-09-15T21:30:14.731465+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:14.731465+0000","last_undegraded":"2024-09-15T21:30:14.731465+0000","last_fullsized":"2024-09-15T21:30:14.731465+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19676,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731407+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:14.731407+0000","last_peered":"2024-09-15T21:30:14.731407+0000","last_clean":"2024-09-15T21:30:14.731407+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:14.731407+0000","last_undegraded":"2024-09-15T21:30:14.731407+0000","last_fullsized":"2024-09-15T21:30:14.731407+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731001+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:14.731001+0000","last_peered":"2024-09-15T21:30:14.731001+0000","last_clean":"2024-09-15T21:30:14.731001+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:14.731001+0000","last_undegraded":"2024-09-15T21:30:14.731001+0000","last_fullsized":"2024-09-15T21:30:14.731001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25233,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731097+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:14.731097+0000","last_peered":"2024-09-15T21:30:14.731097+0000","last_clean":"2024-09-15T21:30:14.731097+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:14.731097+0000","last_undegraded":"2024-09-15T21:30:14.731097+0000","last_fullsized":"2024-09-15T21:30:14.731097+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":123,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731798+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:14.731798+0000","last_peered":"2024-09-15T21:30:14.731798+0000","last_clean":"2024-09-15T21:30:14.731798+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:14.731798+0000","last_undegraded":"2024-09-15T21:30:14.731798+0000","last_fullsized":"2024-09-15T21:30:14.731798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19487,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732655+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:14.732655+0000","last_peered":"2024-09-15T21:30:14.732655+0000","last_clean":"2024-09-15T21:30:14.732655+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:14.732655+0000","last_undegraded":"2024-09-15T21:30:14.732655+0000","last_fullsized":"2024-09-15T21:30:14.732655+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731964+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:14.731964+0000","last_peered":"2024-09-15T21:30:14.731964+0000","last_clean":"2024-09-15T21:30:14.731964+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:14.731964+0000","last_undegraded":"2024-09-15T21:30:14.731964+0000","last_fullsized":"2024-09-15T21:30:14.731964+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25360,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732210+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:14.732210+0000","last_peered":"2024-09-15T21:30:14.732210+0000","last_clean":"2024-09-15T21:30:14.732210+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:14.732210+0000","last_undegraded":"2024-09-15T21:30:14.732210+0000","last_fullsized":"2024-09-15T21:30:14.732210+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":24115,"reported_epoch":66,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:35.320259+0000","last_change":"2024-09-15T21:30:12.884321+0000","last_active":"2024-09-15T21:30:35.320259+0000","last_peered":"2024-09-15T21:30:35.320259+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:35.320259+0000","last_undegraded":"2024-09-15T21:18:43.487554+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":43,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.731973+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:14.731973+0000","last_peered":"2024-09-15T21:30:14.731973+0000","last_clean":"2024-09-15T21:30:14.731973+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:14.731973+0000","last_undegraded":"2024-09-15T21:30:14.731973+0000","last_fullsized":"2024-09-15T21:30:14.731973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732529+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:14.732529+0000","last_peered":"2024-09-15T21:30:14.732529+0000","last_clean":"2024-09-15T21:30:14.732529+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:14.732529+0000","last_undegraded":"2024-09-15T21:30:14.732529+0000","last_fullsized":"2024-09-15T21:30:14.732529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25747,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732068+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:14.732068+0000","last_peered":"2024-09-15T21:30:14.732068+0000","last_clean":"2024-09-15T21:30:14.732068+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:14.732068+0000","last_undegraded":"2024-09-15T21:30:14.732068+0000","last_fullsized":"2024-09-15T21:30:14.732068+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39982,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68456448,"data_stored":68449490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953635,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490944,"kb_used_data":53928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93880896,"statfs":{"total":96636764160,"available":96134037504,"internally_reserved":0,"allocated":55222272,"data_stored":54849727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469552,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902288,"statfs":{"total":96636764160,"available":96155942912,"internally_reserved":0,"allocated":557056,"data_stored":184511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":480243372},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54669312,"data_stored":54665833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:43.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:43.415 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:43.416 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:43.434 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:30:42.655450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1261732,"kb_used_data":44612,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281853788,"statfs":{"total":289910292480,"available":288618278912,"internally_reserved":0,"allocated":45682688,"data_stored":44567147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25832,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:40.816458+0000","last_peered":"2024-09-15T21:30:40.816458+0000","last_clean":"2024-09-15T21:30:40.816458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:40.816458+0000","last_undegraded":"2024-09-15T21:30:40.816458+0000","last_fullsized":"2024-09-15T21:30:40.816458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19233,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815971+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:40.815971+0000","last_peered":"2024-09-15T21:30:40.815971+0000","last_clean":"2024-09-15T21:30:40.815971+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:40.815971+0000","last_undegraded":"2024-09-15T21:30:40.815971+0000","last_fullsized":"2024-09-15T21:30:40.815971+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19678,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815811+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:40.815811+0000","last_peered":"2024-09-15T21:30:40.815811+0000","last_clean":"2024-09-15T21:30:40.815811+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:40.815811+0000","last_undegraded":"2024-09-15T21:30:40.815811+0000","last_fullsized":"2024-09-15T21:30:40.815811+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815708+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:40.815708+0000","last_peered":"2024-09-15T21:30:40.815708+0000","last_clean":"2024-09-15T21:30:40.815708+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:40.815708+0000","last_undegraded":"2024-09-15T21:30:40.815708+0000","last_fullsized":"2024-09-15T21:30:40.815708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25235,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816032+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:40.816032+0000","last_peered":"2024-09-15T21:30:40.816032+0000","last_clean":"2024-09-15T21:30:40.816032+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:40.816032+0000","last_undegraded":"2024-09-15T21:30:40.816032+0000","last_fullsized":"2024-09-15T21:30:40.816032+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":125,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815972+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:40.815972+0000","last_peered":"2024-09-15T21:30:40.815972+0000","last_clean":"2024-09-15T21:30:40.815972+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:40.815972+0000","last_undegraded":"2024-09-15T21:30:40.815972+0000","last_fullsized":"2024-09-15T21:30:40.815972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19489,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816319+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:40.816319+0000","last_peered":"2024-09-15T21:30:40.816319+0000","last_clean":"2024-09-15T21:30:40.816319+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:40.816319+0000","last_undegraded":"2024-09-15T21:30:40.816319+0000","last_fullsized":"2024-09-15T21:30:40.816319+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816384+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:40.816384+0000","last_peered":"2024-09-15T21:30:40.816384+0000","last_clean":"2024-09-15T21:30:40.816384+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:40.816384+0000","last_undegraded":"2024-09-15T21:30:40.816384+0000","last_fullsized":"2024-09-15T21:30:40.816384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25362,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816153+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:40.816153+0000","last_peered":"2024-09-15T21:30:40.816153+0000","last_clean":"2024-09-15T21:30:40.816153+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:40.816153+0000","last_undegraded":"2024-09-15T21:30:40.816153+0000","last_fullsized":"2024-09-15T21:30:40.816153+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19048,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-15T21:30:40.021003+0000","last_change":"2024-09-15T21:30:41.342320+0000","last_active":"2024-09-15T21:30:41.342320+0000","last_peered":"2024-09-15T21:30:40.021003+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:40.021003+0000","last_undegraded":"2024-09-15T21:30:40.021003+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816098+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:40.816098+0000","last_peered":"2024-09-15T21:30:40.816098+0000","last_clean":"2024-09-15T21:30:40.816098+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:40.816098+0000","last_undegraded":"2024-09-15T21:30:40.816098+0000","last_fullsized":"2024-09-15T21:30:40.816098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816189+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:40.816189+0000","last_peered":"2024-09-15T21:30:40.816189+0000","last_clean":"2024-09-15T21:30:40.816189+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:40.816189+0000","last_undegraded":"2024-09-15T21:30:40.816189+0000","last_fullsized":"2024-09-15T21:30:40.816189+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25749,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816269+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:40.816269+0000","last_peered":"2024-09-15T21:30:40.816269+0000","last_clean":"2024-09-15T21:30:40.816269+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:40.816269+0000","last_undegraded":"2024-09-15T21:30:40.816269+0000","last_fullsized":"2024-09-15T21:30:40.816269+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37507072,"data_stored":37500114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467432,"kb_used_data":30064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93904408,"statfs":{"total":96636764160,"available":96158113792,"internally_reserved":0,"allocated":30785536,"data_stored":30412991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473724,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898116,"statfs":{"total":96636764160,"available":96151670784,"internally_reserved":0,"allocated":565248,"data_stored":187241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23719936,"data_stored":23716457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:43.436 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:43.726 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:43.726 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:43.742 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:30:42.655450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1261732,"kb_used_data":44612,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281853788,"statfs":{"total":289910292480,"available":288618278912,"internally_reserved":0,"allocated":45682688,"data_stored":44567147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25832,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:40.816458+0000","last_peered":"2024-09-15T21:30:40.816458+0000","last_clean":"2024-09-15T21:30:40.816458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:40.816458+0000","last_undegraded":"2024-09-15T21:30:40.816458+0000","last_fullsized":"2024-09-15T21:30:40.816458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19233,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815971+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:40.815971+0000","last_peered":"2024-09-15T21:30:40.815971+0000","last_clean":"2024-09-15T21:30:40.815971+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:40.815971+0000","last_undegraded":"2024-09-15T21:30:40.815971+0000","last_fullsized":"2024-09-15T21:30:40.815971+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19678,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815811+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:40.815811+0000","last_peered":"2024-09-15T21:30:40.815811+0000","last_clean":"2024-09-15T21:30:40.815811+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:40.815811+0000","last_undegraded":"2024-09-15T21:30:40.815811+0000","last_fullsized":"2024-09-15T21:30:40.815811+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815708+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:40.815708+0000","last_peered":"2024-09-15T21:30:40.815708+0000","last_clean":"2024-09-15T21:30:40.815708+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:40.815708+0000","last_undegraded":"2024-09-15T21:30:40.815708+0000","last_fullsized":"2024-09-15T21:30:40.815708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25235,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816032+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:40.816032+0000","last_peered":"2024-09-15T21:30:40.816032+0000","last_clean":"2024-09-15T21:30:40.816032+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:40.816032+0000","last_undegraded":"2024-09-15T21:30:40.816032+0000","last_fullsized":"2024-09-15T21:30:40.816032+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":125,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815972+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:40.815972+0000","last_peered":"2024-09-15T21:30:40.815972+0000","last_clean":"2024-09-15T21:30:40.815972+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:40.815972+0000","last_undegraded":"2024-09-15T21:30:40.815972+0000","last_fullsized":"2024-09-15T21:30:40.815972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19489,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816319+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:40.816319+0000","last_peered":"2024-09-15T21:30:40.816319+0000","last_clean":"2024-09-15T21:30:40.816319+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:40.816319+0000","last_undegraded":"2024-09-15T21:30:40.816319+0000","last_fullsized":"2024-09-15T21:30:40.816319+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816384+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:40.816384+0000","last_peered":"2024-09-15T21:30:40.816384+0000","last_clean":"2024-09-15T21:30:40.816384+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:40.816384+0000","last_undegraded":"2024-09-15T21:30:40.816384+0000","last_fullsized":"2024-09-15T21:30:40.816384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25362,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816153+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:40.816153+0000","last_peered":"2024-09-15T21:30:40.816153+0000","last_clean":"2024-09-15T21:30:40.816153+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:40.816153+0000","last_undegraded":"2024-09-15T21:30:40.816153+0000","last_fullsized":"2024-09-15T21:30:40.816153+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19048,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-15T21:30:40.021003+0000","last_change":"2024-09-15T21:30:41.342320+0000","last_active":"2024-09-15T21:30:41.342320+0000","last_peered":"2024-09-15T21:30:40.021003+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:40.021003+0000","last_undegraded":"2024-09-15T21:30:40.021003+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816098+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:40.816098+0000","last_peered":"2024-09-15T21:30:40.816098+0000","last_clean":"2024-09-15T21:30:40.816098+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:40.816098+0000","last_undegraded":"2024-09-15T21:30:40.816098+0000","last_fullsized":"2024-09-15T21:30:40.816098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816189+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:40.816189+0000","last_peered":"2024-09-15T21:30:40.816189+0000","last_clean":"2024-09-15T21:30:40.816189+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:40.816189+0000","last_undegraded":"2024-09-15T21:30:40.816189+0000","last_fullsized":"2024-09-15T21:30:40.816189+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25749,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816269+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:40.816269+0000","last_peered":"2024-09-15T21:30:40.816269+0000","last_clean":"2024-09-15T21:30:40.816269+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:40.816269+0000","last_undegraded":"2024-09-15T21:30:40.816269+0000","last_fullsized":"2024-09-15T21:30:40.816269+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37507072,"data_stored":37500114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467432,"kb_used_data":30064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93904408,"statfs":{"total":96636764160,"available":96158113792,"internally_reserved":0,"allocated":30785536,"data_stored":30412991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473724,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898116,"statfs":{"total":96636764160,"available":96151670784,"internally_reserved":0,"allocated":565248,"data_stored":187241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23719936,"data_stored":23716457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:43.744 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:30:43.744 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "54'18378", 'reported_seq': 19048, 'reported_epoch': 67, 'state': 'peering', 'last_fresh': '2024-09-15T21:30:40.021003+0000', 'last_change': '2024-09-15T21:30:41.342320+0000', 'last_active': '2024-09-15T21:30:41.342320+0000', 'last_peered': '2024-09-15T21:30:40.021003+0000', 'last_clean': '2024-09-15T21:18:41.339576+0000', 'last_became_active': '2024-09-15T21:18:43.895424+0000', 'last_became_peered': '2024-09-15T21:18:43.895424+0000', 'last_unstale': '2024-09-15T21:30:40.021003+0000', 'last_undegraded': '2024-09-15T21:30:40.021003+0000', 'last_fullsized': '2024-09-15T21:18:43.370275+0000', 'mapping_epoch': 67, 'log_start': "46'8300", 'ondisk_log_start': "46'8300", 'created': 17, 'last_epoch_clean': 40, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "37'2353", 'last_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:16:54.375167+0000', 'last_clean_scrub_stamp': '2024-09-15T21:18:21.993612+0000', 'objects_scrubbed': 2349, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 283, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16025, 'num_write_kb': 27352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3367, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:30:43.744 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:44.079 INFO:teuthology.orchestra.run.smithi114.stdout:{"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":857,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435121,"num_in_osds":3,"osd_in_since":1726435109,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1292013568,"bytes_avail":288618278912,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"recovering_objects_per_sec":144,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1154,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:16:26:965950+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-15T21:30:06.645568+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:44.100 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:44.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:44.387 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:44.404 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:30:42.655450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1261732,"kb_used_data":44612,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281853788,"statfs":{"total":289910292480,"available":288618278912,"internally_reserved":0,"allocated":45682688,"data_stored":44567147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1245891594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1154,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002208"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25374,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735400+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:14.735400+0000","last_peered":"2024-09-15T21:30:14.735400+0000","last_clean":"2024-09-15T21:30:14.735400+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:14.735400+0000","last_undegraded":"2024-09-15T21:30:14.735400+0000","last_fullsized":"2024-09-15T21:30:14.735400+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18780,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732668+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:14.732668+0000","last_peered":"2024-09-15T21:30:14.732668+0000","last_clean":"2024-09-15T21:30:14.732668+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:14.732668+0000","last_undegraded":"2024-09-15T21:30:14.732668+0000","last_fullsized":"2024-09-15T21:30:14.732668+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25832,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816458+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:40.816458+0000","last_peered":"2024-09-15T21:30:40.816458+0000","last_clean":"2024-09-15T21:30:40.816458+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:40.816458+0000","last_undegraded":"2024-09-15T21:30:40.816458+0000","last_fullsized":"2024-09-15T21:30:40.816458+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18683,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732336+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:14.732336+0000","last_peered":"2024-09-15T21:30:14.732336+0000","last_clean":"2024-09-15T21:30:14.732336+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:14.732336+0000","last_undegraded":"2024-09-15T21:30:14.732336+0000","last_fullsized":"2024-09-15T21:30:14.732336+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18885,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735224+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:14.735224+0000","last_peered":"2024-09-15T21:30:14.735224+0000","last_clean":"2024-09-15T21:30:14.735224+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:14.735224+0000","last_undegraded":"2024-09-15T21:30:14.735224+0000","last_fullsized":"2024-09-15T21:30:14.735224+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19233,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815971+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:40.815971+0000","last_peered":"2024-09-15T21:30:40.815971+0000","last_clean":"2024-09-15T21:30:40.815971+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:40.815971+0000","last_undegraded":"2024-09-15T21:30:40.815971+0000","last_fullsized":"2024-09-15T21:30:40.815971+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19678,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815811+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:40.815811+0000","last_peered":"2024-09-15T21:30:40.815811+0000","last_clean":"2024-09-15T21:30:40.815811+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:40.815811+0000","last_undegraded":"2024-09-15T21:30:40.815811+0000","last_fullsized":"2024-09-15T21:30:40.815811+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18568,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.734959+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:14.734959+0000","last_peered":"2024-09-15T21:30:14.734959+0000","last_clean":"2024-09-15T21:30:14.734959+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:14.734959+0000","last_undegraded":"2024-09-15T21:30:14.734959+0000","last_fullsized":"2024-09-15T21:30:14.734959+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815708+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:40.815708+0000","last_peered":"2024-09-15T21:30:40.815708+0000","last_clean":"2024-09-15T21:30:40.815708+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:40.815708+0000","last_undegraded":"2024-09-15T21:30:40.815708+0000","last_fullsized":"2024-09-15T21:30:40.815708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25235,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816032+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:40.816032+0000","last_peered":"2024-09-15T21:30:40.816032+0000","last_clean":"2024-09-15T21:30:40.816032+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:40.816032+0000","last_undegraded":"2024-09-15T21:30:40.816032+0000","last_fullsized":"2024-09-15T21:30:40.816032+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732912+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:14.732912+0000","last_peered":"2024-09-15T21:30:14.732912+0000","last_clean":"2024-09-15T21:30:14.732912+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:14.732912+0000","last_undegraded":"2024-09-15T21:30:14.732912+0000","last_fullsized":"2024-09-15T21:30:14.732912+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24895,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.814682+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:14.814682+0000","last_peered":"2024-09-15T21:30:14.814682+0000","last_clean":"2024-09-15T21:30:14.814682+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:14.814682+0000","last_undegraded":"2024-09-15T21:30:14.814682+0000","last_fullsized":"2024-09-15T21:30:14.814682+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":145,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.735596+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:14.735596+0000","last_peered":"2024-09-15T21:30:14.735596+0000","last_clean":"2024-09-15T21:30:14.735596+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:14.735596+0000","last_undegraded":"2024-09-15T21:30:14.735596+0000","last_fullsized":"2024-09-15T21:30:14.735596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24981,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.733049+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:14.733049+0000","last_peered":"2024-09-15T21:30:14.733049+0000","last_clean":"2024-09-15T21:30:14.733049+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:14.733049+0000","last_undegraded":"2024-09-15T21:30:14.733049+0000","last_fullsized":"2024-09-15T21:30:14.733049+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":125,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.815972+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:40.815972+0000","last_peered":"2024-09-15T21:30:40.815972+0000","last_clean":"2024-09-15T21:30:40.815972+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:40.815972+0000","last_undegraded":"2024-09-15T21:30:40.815972+0000","last_fullsized":"2024-09-15T21:30:40.815972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25419,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732196+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:14.732196+0000","last_peered":"2024-09-15T21:30:14.732196+0000","last_clean":"2024-09-15T21:30:14.732196+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:14.732196+0000","last_undegraded":"2024-09-15T21:30:14.732196+0000","last_fullsized":"2024-09-15T21:30:14.732196+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19489,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816319+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:40.816319+0000","last_peered":"2024-09-15T21:30:40.816319+0000","last_clean":"2024-09-15T21:30:40.816319+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:40.816319+0000","last_undegraded":"2024-09-15T21:30:40.816319+0000","last_fullsized":"2024-09-15T21:30:40.816319+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816384+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:40.816384+0000","last_peered":"2024-09-15T21:30:40.816384+0000","last_clean":"2024-09-15T21:30:40.816384+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:40.816384+0000","last_undegraded":"2024-09-15T21:30:40.816384+0000","last_fullsized":"2024-09-15T21:30:40.816384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25362,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816153+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:40.816153+0000","last_peered":"2024-09-15T21:30:40.816153+0000","last_clean":"2024-09-15T21:30:40.816153+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:40.816153+0000","last_undegraded":"2024-09-15T21:30:40.816153+0000","last_fullsized":"2024-09-15T21:30:40.816153+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.732827+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:14.732827+0000","last_peered":"2024-09-15T21:30:14.732827+0000","last_clean":"2024-09-15T21:30:14.732827+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:14.732827+0000","last_undegraded":"2024-09-15T21:30:14.732827+0000","last_fullsized":"2024-09-15T21:30:14.732827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19048,"reported_epoch":67,"state":"peering","last_fresh":"2024-09-15T21:30:40.021003+0000","last_change":"2024-09-15T21:30:41.342320+0000","last_active":"2024-09-15T21:30:41.342320+0000","last_peered":"2024-09-15T21:30:40.021003+0000","last_clean":"2024-09-15T21:18:41.339576+0000","last_became_active":"2024-09-15T21:18:43.895424+0000","last_became_peered":"2024-09-15T21:18:43.895424+0000","last_unstale":"2024-09-15T21:30:40.021003+0000","last_undegraded":"2024-09-15T21:30:40.021003+0000","last_fullsized":"2024-09-15T21:18:43.370275+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816098+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:40.816098+0000","last_peered":"2024-09-15T21:30:40.816098+0000","last_clean":"2024-09-15T21:30:40.816098+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:40.816098+0000","last_undegraded":"2024-09-15T21:30:40.816098+0000","last_fullsized":"2024-09-15T21:30:40.816098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":120,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816189+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:40.816189+0000","last_peered":"2024-09-15T21:30:40.816189+0000","last_clean":"2024-09-15T21:30:40.816189+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:40.816189+0000","last_undegraded":"2024-09-15T21:30:40.816189+0000","last_fullsized":"2024-09-15T21:30:40.816189+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25749,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:30:40.816269+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:40.816269+0000","last_peered":"2024-09-15T21:30:40.816269+0000","last_clean":"2024-09-15T21:30:40.816269+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:40.816269+0000","last_undegraded":"2024-09-15T21:30:40.816269+0000","last_fullsized":"2024-09-15T21:30:40.816269+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37507072,"data_stored":37500114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953636,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320576,"kb_used_data":13996,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94051264,"statfs":{"total":96636764160,"available":96308494336,"internally_reserved":0,"allocated":14331904,"data_stored":13966915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149861,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467432,"kb_used_data":30064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93904408,"statfs":{"total":96636764160,"available":96158113792,"internally_reserved":0,"allocated":30785536,"data_stored":30412991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":447475377},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473724,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898116,"statfs":{"total":96636764160,"available":96151670784,"internally_reserved":0,"allocated":565248,"data_stored":187241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23719936,"data_stored":23716457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13783040,"data_stored":13783040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:47.406 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:47.718 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:47.719 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:47.736 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-15T21:30:46.656347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002068"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:47.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-15T21:30:47.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-15T21:30:47.737 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:48.037 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:48.037 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:48.055 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-15T21:30:46.656347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002068"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:48.057 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:48.351 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:48.352 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:48.369 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-15T21:30:46.656347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002068"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:48.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-15T21:30:48.370 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:30:48.380 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:30:48.381 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:48.683 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:48.701 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-15T21:30:48.656946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002088"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:48.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:49.002 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:49.022 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-15T21:30:48.656946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002088"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:49.024 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:30:49.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:49.327 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:49.346 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-15T21:30:48.656946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002088"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18889,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827122+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827122+0000","last_peered":"2024-09-15T21:30:41.827122+0000","last_clean":"2024-09-15T21:30:41.827122+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827122+0000","last_undegraded":"2024-09-15T21:30:41.827122+0000","last_fullsized":"2024-09-15T21:30:41.827122+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953637,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:49.348 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:30:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:49.650 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":68,"fsid":"84ba6ec4-065b-47f6-87f1-d15da93be3d3","created":"2024-09-15T21:16:26.966560+0000","modified":"2024-09-15T21:30:41.810597+0000","last_up_change":"2024-09-15T21:18:41.342812+0000","last_in_change":"2024-09-15T21:18:29.636613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:16:42.738751+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-15T21:16:54.321157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3944bfc1-02b1-40e5-9748-a6b5781fcfd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":22,"up_from":26,"up_thru":66,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":490656443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6805","nonce":490656443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6807","nonce":490656443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":490656443}]},"public_addr":"172.21.15.114:6804/490656443","cluster_addr":"172.21.15.114:6805/490656443","heartbeat_back_addr":"172.21.15.114:6807/490656443","heartbeat_front_addr":"172.21.15.114:6806/490656443","state":["exists","up"]},{"osd":1,"uuid":"8e707e07-76a7-4d0b-b923-42e77df0f4eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":27,"up_from":42,"up_thru":67,"down_at":28,"lost_at":28,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1392950530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6809","nonce":1392950530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6811","nonce":1392950530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1392950530}]},"public_addr":"172.21.15.114:6808/1392950530","cluster_addr":"172.21.15.114:6809/1392950530","heartbeat_back_addr":"172.21.15.114:6811/1392950530","heartbeat_front_addr":"172.21.15.114:6810/1392950530","state":["exists","up"]},{"osd":2,"uuid":"7647c42f-573c-4e9b-a6e4-b65509427795","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":10,"last_clean_end":19,"up_from":32,"up_thru":57,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":42043953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6801","nonce":42043953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6803","nonce":42043953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":42043953}]},"public_addr":"172.21.15.114:6800/42043953","cluster_addr":"172.21.15.114:6801/42043953","heartbeat_back_addr":"172.21.15.114:6803/42043953","heartbeat_front_addr":"172.21.15.114:6802/42043953","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.083905+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:34.869670+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:16:35.523227+0000","dead_epoch":20}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:30:50.669 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:30:50.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:30:50.831 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:30:50.831 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:30:50.831 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:30:50.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:30:51.142 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:30:51.160 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:30:51.160 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:30:51.299 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:30:51.300 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:30:51.300 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:30:51.315 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:30:51.592 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:30:51.609 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:30:51.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:30:51.953 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-15T21:30:51.954 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:30:51.954 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-15T21:30:51.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:30:52.287 INFO:teuthology.orchestra.run.smithi114.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:30:52.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:30:52.583 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:30:52.600 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-15T21:30:50.657406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222604,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892916,"statfs":{"total":289910292480,"available":288658345984,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.843161"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25378,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.828011+0000","last_change":"2024-09-15T21:27:53.880920+0000","last_active":"2024-09-15T21:30:41.828011+0000","last_peered":"2024-09-15T21:30:41.828011+0000","last_clean":"2024-09-15T21:30:41.828011+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:41.828011+0000","last_undegraded":"2024-09-15T21:30:41.828011+0000","last_fullsized":"2024-09-15T21:30:41.828011+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:55.424300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18784,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822721+0000","last_change":"2024-09-15T21:18:04.864066+0000","last_active":"2024-09-15T21:30:41.822721+0000","last_peered":"2024-09-15T21:30:41.822721+0000","last_clean":"2024-09-15T21:30:41.822721+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:41.822721+0000","last_undegraded":"2024-09-15T21:30:41.822721+0000","last_fullsized":"2024-09-15T21:30:41.822721+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1275","last_scrub_stamp":"2024-09-15T21:18:04.863968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:04.863968+0000","objects_scrubbed":1268,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:28.177263+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25834,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823205+0000","last_change":"2024-09-15T21:20:16.453134+0000","last_active":"2024-09-15T21:30:41.823205+0000","last_peered":"2024-09-15T21:30:41.823205+0000","last_clean":"2024-09-15T21:30:41.823205+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:30:41.823205+0000","last_undegraded":"2024-09-15T21:30:41.823205+0000","last_fullsized":"2024-09-15T21:30:41.823205+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1305","last_scrub_stamp":"2024-09-15T21:18:06.394090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:06.394090+0000","objects_scrubbed":1273,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:42.516117+0000","scrub_duration":683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18687,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822919+0000","last_change":"2024-09-15T21:17:46.447397+0000","last_active":"2024-09-15T21:30:41.822919+0000","last_peered":"2024-09-15T21:30:41.822919+0000","last_clean":"2024-09-15T21:30:41.822919+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:41.822919+0000","last_undegraded":"2024-09-15T21:30:41.822919+0000","last_fullsized":"2024-09-15T21:30:41.822919+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:16:54.375167+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:42.332152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18890,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827271+0000","last_change":"2024-09-15T21:18:18.536399+0000","last_active":"2024-09-15T21:30:41.827271+0000","last_peered":"2024-09-15T21:30:41.827271+0000","last_clean":"2024-09-15T21:30:41.827271+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:41.827271+0000","last_undegraded":"2024-09-15T21:30:41.827271+0000","last_fullsized":"2024-09-15T21:30:41.827271+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2177","last_scrub_stamp":"2024-09-15T21:18:18.536295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:18.536295+0000","objects_scrubbed":2135,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:18:00.860120+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19235,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820995+0000","last_change":"2024-09-15T21:29:15.006268+0000","last_active":"2024-09-15T21:30:41.820995+0000","last_peered":"2024-09-15T21:30:41.820995+0000","last_clean":"2024-09-15T21:30:41.820995+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:30:41.820995+0000","last_undegraded":"2024-09-15T21:30:41.820995+0000","last_fullsized":"2024-09-15T21:30:41.820995+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2630","last_scrub_stamp":"2024-09-15T21:18:26.026796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.026796+0000","objects_scrubbed":2570,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:58.254105+0000","scrub_duration":1295,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19680,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820887+0000","last_change":"2024-09-15T21:29:46.844400+0000","last_active":"2024-09-15T21:30:41.820887+0000","last_peered":"2024-09-15T21:30:41.820887+0000","last_clean":"2024-09-15T21:30:41.820887+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:30:41.820887+0000","last_undegraded":"2024-09-15T21:30:41.820887+0000","last_fullsized":"2024-09-15T21:30:41.820887+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2404","last_scrub_stamp":"2024-09-15T21:18:21.588315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.588315+0000","objects_scrubbed":2358,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:10.493501+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18572,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.826927+0000","last_change":"2024-09-15T21:18:19.639577+0000","last_active":"2024-09-15T21:30:41.826927+0000","last_peered":"2024-09-15T21:30:41.826927+0000","last_clean":"2024-09-15T21:30:41.826927+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:41.826927+0000","last_undegraded":"2024-09-15T21:30:41.826927+0000","last_fullsized":"2024-09-15T21:30:41.826927+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2264","last_scrub_stamp":"2024-09-15T21:18:19.639517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:19.639517+0000","objects_scrubbed":2213,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:56.121079+0000","scrub_duration":895,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820597+0000","last_change":"2024-09-15T21:18:43.396647+0000","last_active":"2024-09-15T21:30:41.820597+0000","last_peered":"2024-09-15T21:30:41.820597+0000","last_clean":"2024-09-15T21:30:41.820597+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:41.820597+0000","last_undegraded":"2024-09-15T21:30:41.820597+0000","last_fullsized":"2024-09-15T21:30:41.820597+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:42.580721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25237,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821214+0000","last_change":"2024-09-15T21:23:34.695762+0000","last_active":"2024-09-15T21:30:41.821214+0000","last_peered":"2024-09-15T21:30:41.821214+0000","last_clean":"2024-09-15T21:30:41.821214+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:41.821214+0000","last_undegraded":"2024-09-15T21:30:41.821214+0000","last_fullsized":"2024-09-15T21:30:41.821214+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1431","last_scrub_stamp":"2024-09-15T21:18:08.359695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:08.359695+0000","objects_scrubbed":1407,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:03.858278+0000","scrub_duration":575,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821845+0000","last_change":"2024-09-15T21:17:46.446029+0000","last_active":"2024-09-15T21:30:41.821845+0000","last_peered":"2024-09-15T21:30:41.821845+0000","last_clean":"2024-09-15T21:30:41.821845+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:41.821845+0000","last_undegraded":"2024-09-15T21:30:41.821845+0000","last_fullsized":"2024-09-15T21:30:41.821845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:48:01.793291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24899,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827261+0000","last_change":"2024-09-15T21:30:14.814558+0000","last_active":"2024-09-15T21:30:41.827261+0000","last_peered":"2024-09-15T21:30:41.827261+0000","last_clean":"2024-09-15T21:30:41.827261+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:41.827261+0000","last_undegraded":"2024-09-15T21:30:41.827261+0000","last_fullsized":"2024-09-15T21:30:41.827261+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2513","last_scrub_stamp":"2024-09-15T21:18:24.693269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:24.693269+0000","objects_scrubbed":2466,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:41:30.895131+0000","scrub_duration":971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":149,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.827012+0000","last_change":"2024-09-15T21:18:43.896077+0000","last_active":"2024-09-15T21:30:41.827012+0000","last_peered":"2024-09-15T21:30:41.827012+0000","last_clean":"2024-09-15T21:30:41.827012+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:41.827012+0000","last_undegraded":"2024-09-15T21:30:41.827012+0000","last_fullsized":"2024-09-15T21:30:41.827012+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:39.090100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24985,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822101+0000","last_change":"2024-09-15T21:26:58.835453+0000","last_active":"2024-09-15T21:30:41.822101+0000","last_peered":"2024-09-15T21:30:41.822101+0000","last_clean":"2024-09-15T21:30:41.822101+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:41.822101+0000","last_undegraded":"2024-09-15T21:30:41.822101+0000","last_fullsized":"2024-09-15T21:30:41.822101+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1469","last_scrub_stamp":"2024-09-15T21:18:09.166204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:09.166204+0000","objects_scrubbed":1444,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:06:56.362033+0000","scrub_duration":427,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":127,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821040+0000","last_change":"2024-09-15T21:18:43.469162+0000","last_active":"2024-09-15T21:30:41.821040+0000","last_peered":"2024-09-15T21:30:41.821040+0000","last_clean":"2024-09-15T21:30:41.821040+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:41.821040+0000","last_undegraded":"2024-09-15T21:30:41.821040+0000","last_fullsized":"2024-09-15T21:30:41.821040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:36:13.159025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25423,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820671+0000","last_change":"2024-09-15T21:28:20.795228+0000","last_active":"2024-09-15T21:30:41.820671+0000","last_peered":"2024-09-15T21:30:41.820671+0000","last_clean":"2024-09-15T21:30:41.820671+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:41.820671+0000","last_undegraded":"2024-09-15T21:30:41.820671+0000","last_fullsized":"2024-09-15T21:30:41.820671+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'1630","last_scrub_stamp":"2024-09-15T21:18:10.504173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:10.504173+0000","objects_scrubbed":1595,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:24:12.174638+0000","scrub_duration":739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19491,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822689+0000","last_change":"2024-09-15T21:28:48.018581+0000","last_active":"2024-09-15T21:30:41.822689+0000","last_peered":"2024-09-15T21:30:41.822689+0000","last_clean":"2024-09-15T21:30:41.822689+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:30:41.822689+0000","last_undegraded":"2024-09-15T21:30:41.822689+0000","last_fullsized":"2024-09-15T21:30:41.822689+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2736","last_scrub_stamp":"2024-09-15T21:18:26.850672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:26.850672+0000","objects_scrubbed":2691,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:05.333063+0000","scrub_duration":1139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821532+0000","last_change":"2024-09-15T21:18:43.606386+0000","last_active":"2024-09-15T21:30:41.821532+0000","last_peered":"2024-09-15T21:30:41.821532+0000","last_clean":"2024-09-15T21:30:41.821532+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:41.821532+0000","last_undegraded":"2024-09-15T21:30:41.821532+0000","last_fullsized":"2024-09-15T21:30:41.821532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:58.451419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25364,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821918+0000","last_change":"2024-09-15T21:25:03.819450+0000","last_active":"2024-09-15T21:30:41.821918+0000","last_peered":"2024-09-15T21:30:41.821918+0000","last_clean":"2024-09-15T21:30:41.821918+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:30:41.821918+0000","last_undegraded":"2024-09-15T21:30:41.821918+0000","last_fullsized":"2024-09-15T21:30:41.821918+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2446","last_scrub_stamp":"2024-09-15T21:18:23.510869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:23.510869+0000","objects_scrubbed":2407,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:35.938106+0000","scrub_duration":755,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.823483+0000","last_change":"2024-09-15T21:17:46.444131+0000","last_active":"2024-09-15T21:30:41.823483+0000","last_peered":"2024-09-15T21:30:41.823483+0000","last_clean":"2024-09-15T21:30:41.823483+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:41.823483+0000","last_undegraded":"2024-09-15T21:30:41.823483+0000","last_fullsized":"2024-09-15T21:30:41.823483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:37.484013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19057,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.993086+0000","last_change":"2024-09-15T21:30:41.993001+0000","last_active":"2024-09-15T21:30:41.993086+0000","last_peered":"2024-09-15T21:30:41.993086+0000","last_clean":"2024-09-15T21:30:41.993086+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:41.993086+0000","last_undegraded":"2024-09-15T21:30:41.993086+0000","last_fullsized":"2024-09-15T21:30:41.993086+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2353","last_scrub_stamp":"2024-09-15T21:18:21.993612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:21.993612+0000","objects_scrubbed":2349,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:28.971419+0000","scrub_duration":283,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":117,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.821690+0000","last_change":"2024-09-15T21:18:43.606851+0000","last_active":"2024-09-15T21:30:41.821690+0000","last_peered":"2024-09-15T21:30:41.821690+0000","last_clean":"2024-09-15T21:30:41.821690+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:41.821690+0000","last_undegraded":"2024-09-15T21:30:41.821690+0000","last_fullsized":"2024-09-15T21:30:41.821690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:08.571156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.822427+0000","last_change":"2024-09-15T21:18:43.382288+0000","last_active":"2024-09-15T21:30:41.822427+0000","last_peered":"2024-09-15T21:30:41.822427+0000","last_clean":"2024-09-15T21:30:41.822427+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:41.822427+0000","last_undegraded":"2024-09-15T21:30:41.822427+0000","last_fullsized":"2024-09-15T21:30:41.822427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:43.325385+0000","last_clean_scrub_stamp":"2024-09-15T21:16:43.325385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:26:49.150987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"54'18618","reported_seq":25751,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:41.820844+0000","last_change":"2024-09-15T21:22:00.580642+0000","last_active":"2024-09-15T21:30:41.820844+0000","last_peered":"2024-09-15T21:30:41.820844+0000","last_clean":"2024-09-15T21:30:41.820844+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:41.820844+0000","last_undegraded":"2024-09-15T21:30:41.820844+0000","last_fullsized":"2024-09-15T21:30:41.820844+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2344","last_scrub_stamp":"2024-09-15T21:18:20.389703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:16:54.375167+0000","last_clean_scrub_stamp":"2024-09-15T21:18:20.389703+0000","objects_scrubbed":2329,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:02.406660+0000","scrub_duration":655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953638,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930104,"statfs":{"total":96636764160,"available":96184426496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626581,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473736,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898104,"statfs":{"total":96636764160,"available":96151658496,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-15T21:16:54.375167+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-15T21:18:04.863968+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-15T21:18:06.394090+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=6, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-15T21:16:54.375167+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=54, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-15T21:18:18.536295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=18, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-15T21:18:26.026796+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.605 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-15T21:18:21.588315+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-15T21:18:19.639517+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=19, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:18:08.359695+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=8, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:18:24.693269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=24, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:18:09.166204+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=9, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.606 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:18:10.504173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=10, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:18:26.850672+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:18:23.510869+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=23, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:18:21.993612+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.607 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-15T21:16:43.325385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=16, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.608 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:18:20.389703+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=18, tm_sec=20, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=30, tm_sec=49, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:30:52.608 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:31:12.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-15T21:31:12.918 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-15T21:31:12.933 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-15T21:31:12.663400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267780,"num_write_kb":478547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40566,"num_bytes_recovered":44572691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222584,"kb_used_data":1672,"kb_used_omap":12,"kb_used_meta":1220851,"kb_avail":281892936,"statfs":{"total":289910292480,"available":288658366464,"internally_reserved":0,"allocated":1712128,"data_stored":569214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13302,"internal_metadata":1250151434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"2.f","version":"54'18512","reported_seq":25387,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.357281+0000","last_change":"2024-09-15T21:30:54.357115+0000","last_active":"2024-09-15T21:30:54.357281+0000","last_peered":"2024-09-15T21:30:54.357281+0000","last_clean":"2024-09-15T21:30:54.357281+0000","last_became_active":"2024-09-15T21:27:53.807197+0000","last_became_peered":"2024-09-15T21:27:53.807197+0000","last_unstale":"2024-09-15T21:30:54.357281+0000","last_undegraded":"2024-09-15T21:30:54.357281+0000","last_fullsized":"2024-09-15T21:30:54.357281+0000","mapping_epoch":55,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18512","last_scrub_stamp":"2024-09-15T21:30:54.356921+0000","last_deep_scrub":"54'18512","last_deep_scrub_stamp":"2024-09-15T21:30:54.356921+0000","last_clean_scrub_stamp":"2024-09-15T21:30:54.356921+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:24.835863+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18512,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"54'18640","reported_seq":18793,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.428486+0000","last_change":"2024-09-15T21:30:55.428317+0000","last_active":"2024-09-15T21:30:55.428486+0000","last_peered":"2024-09-15T21:30:55.428486+0000","last_clean":"2024-09-15T21:30:55.428486+0000","last_became_active":"2024-09-15T21:17:46.447003+0000","last_became_peered":"2024-09-15T21:17:46.447003+0000","last_unstale":"2024-09-15T21:30:55.428486+0000","last_undegraded":"2024-09-15T21:30:55.428486+0000","last_fullsized":"2024-09-15T21:30:55.428486+0000","mapping_epoch":36,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18640","last_scrub_stamp":"2024-09-15T21:30:55.428107+0000","last_deep_scrub":"54'18640","last_deep_scrub_stamp":"2024-09-15T21:30:55.428107+0000","last_clean_scrub_stamp":"2024-09-15T21:30:55.428107+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:05.289999+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17366,"num_write_kb":32196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"54'18870","reported_seq":25842,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:31:04.263766+0000","last_change":"2024-09-15T21:31:04.263766+0000","last_active":"2024-09-15T21:31:04.263766+0000","last_peered":"2024-09-15T21:31:04.263766+0000","last_clean":"2024-09-15T21:31:04.263766+0000","last_became_active":"2024-09-15T21:20:16.434043+0000","last_became_peered":"2024-09-15T21:20:16.434043+0000","last_unstale":"2024-09-15T21:31:04.263766+0000","last_undegraded":"2024-09-15T21:31:04.263766+0000","last_fullsized":"2024-09-15T21:31:04.263766+0000","mapping_epoch":45,"log_start":"46'8800","ondisk_log_start":"46'8800","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18870","last_scrub_stamp":"2024-09-15T21:31:04.263583+0000","last_deep_scrub":"54'18870","last_deep_scrub_stamp":"2024-09-15T21:31:04.263583+0000","last_clean_scrub_stamp":"2024-09-15T21:31:04.263583+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:06:21.900197+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17593,"num_write_kb":32637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3393,"num_bytes_recovered":13893632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"54'18570","reported_seq":18696,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:57.391864+0000","last_change":"2024-09-15T21:30:57.391696+0000","last_active":"2024-09-15T21:30:57.391864+0000","last_peered":"2024-09-15T21:30:57.391864+0000","last_clean":"2024-09-15T21:30:57.391864+0000","last_became_active":"2024-09-15T21:17:46.447071+0000","last_became_peered":"2024-09-15T21:17:46.447071+0000","last_unstale":"2024-09-15T21:30:57.391864+0000","last_undegraded":"2024-09-15T21:30:57.391864+0000","last_fullsized":"2024-09-15T21:30:57.391864+0000","mapping_epoch":36,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18570","last_scrub_stamp":"2024-09-15T21:30:57.391509+0000","last_deep_scrub":"54'18570","last_deep_scrub_stamp":"2024-09-15T21:30:57.391509+0000","last_clean_scrub_stamp":"2024-09-15T21:30:57.391509+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:47:04.362474+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18570,"num_write_kb":37140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"54'18723","reported_seq":18898,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:53.341374+0000","last_change":"2024-09-15T21:30:53.341374+0000","last_active":"2024-09-15T21:30:53.341374+0000","last_peered":"2024-09-15T21:30:53.341374+0000","last_clean":"2024-09-15T21:30:53.341374+0000","last_became_active":"2024-09-15T21:17:45.436876+0000","last_became_peered":"2024-09-15T21:17:45.436876+0000","last_unstale":"2024-09-15T21:30:53.341374+0000","last_undegraded":"2024-09-15T21:30:53.341374+0000","last_fullsized":"2024-09-15T21:30:53.341374+0000","mapping_epoch":35,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18723","last_scrub_stamp":"2024-09-15T21:30:53.340633+0000","last_deep_scrub":"54'18723","last_deep_scrub_stamp":"2024-09-15T21:30:53.340633+0000","last_clean_scrub_stamp":"2024-09-15T21:30:53.340633+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:03:30.352040+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16581,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"54'18381","reported_seq":19243,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:31:02.254536+0000","last_change":"2024-09-15T21:31:02.254536+0000","last_active":"2024-09-15T21:31:02.254536+0000","last_peered":"2024-09-15T21:31:02.254536+0000","last_clean":"2024-09-15T21:31:02.254536+0000","last_became_active":"2024-09-15T21:29:14.829468+0000","last_became_peered":"2024-09-15T21:29:14.829468+0000","last_unstale":"2024-09-15T21:31:02.254536+0000","last_undegraded":"2024-09-15T21:31:02.254536+0000","last_fullsized":"2024-09-15T21:31:02.254536+0000","mapping_epoch":61,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18381","last_scrub_stamp":"2024-09-15T21:31:02.254352+0000","last_deep_scrub":"54'18381","last_deep_scrub_stamp":"2024-09-15T21:31:02.254352+0000","last_clean_scrub_stamp":"2024-09-15T21:31:02.254352+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:45.705545+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"54'18904","reported_seq":19688,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:31:03.283808+0000","last_change":"2024-09-15T21:31:03.283808+0000","last_active":"2024-09-15T21:31:03.283808+0000","last_peered":"2024-09-15T21:31:03.283808+0000","last_clean":"2024-09-15T21:31:03.283808+0000","last_became_active":"2024-09-15T21:29:46.749841+0000","last_became_peered":"2024-09-15T21:29:46.749841+0000","last_unstale":"2024-09-15T21:31:03.283808+0000","last_undegraded":"2024-09-15T21:31:03.283808+0000","last_fullsized":"2024-09-15T21:31:03.283808+0000","mapping_epoch":63,"log_start":"46'8900","ondisk_log_start":"46'8900","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18904","last_scrub_stamp":"2024-09-15T21:31:03.283626+0000","last_deep_scrub":"54'18904","last_deep_scrub_stamp":"2024-09-15T21:31:03.283626+0000","last_clean_scrub_stamp":"2024-09-15T21:31:03.283626+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:47.102200+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"54'18404","reported_seq":18580,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.385230+0000","last_change":"2024-09-15T21:30:55.385230+0000","last_active":"2024-09-15T21:30:55.385230+0000","last_peered":"2024-09-15T21:30:55.385230+0000","last_clean":"2024-09-15T21:30:55.385230+0000","last_became_active":"2024-09-15T21:17:45.436722+0000","last_became_peered":"2024-09-15T21:17:45.436722+0000","last_unstale":"2024-09-15T21:30:55.385230+0000","last_undegraded":"2024-09-15T21:30:55.385230+0000","last_fullsized":"2024-09-15T21:30:55.385230+0000","mapping_epoch":35,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18404","last_scrub_stamp":"2024-09-15T21:30:55.385056+0000","last_deep_scrub":"54'18404","last_deep_scrub_stamp":"2024-09-15T21:30:55.385056+0000","last_clean_scrub_stamp":"2024-09-15T21:30:55.385056+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:43.675755+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:53.154094+0000","last_change":"2024-09-15T21:30:53.154094+0000","last_active":"2024-09-15T21:30:53.154094+0000","last_peered":"2024-09-15T21:30:53.154094+0000","last_clean":"2024-09-15T21:30:53.154094+0000","last_became_active":"2024-09-15T21:18:43.395535+0000","last_became_peered":"2024-09-15T21:18:43.395535+0000","last_unstale":"2024-09-15T21:30:53.154094+0000","last_undegraded":"2024-09-15T21:30:53.154094+0000","last_fullsized":"2024-09-15T21:30:53.154094+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:53.153935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:53.153935+0000","last_clean_scrub_stamp":"2024-09-15T21:30:53.153935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:24.871004+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0026175030000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"54'18375","reported_seq":25245,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:58.286818+0000","last_change":"2024-09-15T21:30:58.286818+0000","last_active":"2024-09-15T21:30:58.286818+0000","last_peered":"2024-09-15T21:30:58.286818+0000","last_clean":"2024-09-15T21:30:58.286818+0000","last_became_active":"2024-09-15T21:23:34.664641+0000","last_became_peered":"2024-09-15T21:23:34.664641+0000","last_unstale":"2024-09-15T21:30:58.286818+0000","last_undegraded":"2024-09-15T21:30:58.286818+0000","last_fullsized":"2024-09-15T21:30:58.286818+0000","mapping_epoch":49,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18375","last_scrub_stamp":"2024-09-15T21:30:58.286638+0000","last_deep_scrub":"54'18375","last_deep_scrub_stamp":"2024-09-15T21:30:58.286638+0000","last_clean_scrub_stamp":"2024-09-15T21:30:58.286638+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:27.435840+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16961,"num_write_kb":31108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3340,"num_bytes_recovered":13144064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:53.420238+0000","last_change":"2024-09-15T21:30:53.420238+0000","last_active":"2024-09-15T21:30:53.420238+0000","last_peered":"2024-09-15T21:30:53.420238+0000","last_clean":"2024-09-15T21:30:53.420238+0000","last_became_active":"2024-09-15T21:17:46.445527+0000","last_became_peered":"2024-09-15T21:17:46.445527+0000","last_unstale":"2024-09-15T21:30:53.420238+0000","last_undegraded":"2024-09-15T21:30:53.420238+0000","last_fullsized":"2024-09-15T21:30:53.420238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:53.420036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:53.420036+0000","last_clean_scrub_stamp":"2024-09-15T21:30:53.420036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:30.558380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"54'18034","reported_seq":24907,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:52.354975+0000","last_change":"2024-09-15T21:30:52.354975+0000","last_active":"2024-09-15T21:30:52.354975+0000","last_peered":"2024-09-15T21:30:52.354975+0000","last_clean":"2024-09-15T21:30:52.354975+0000","last_became_active":"2024-09-15T21:30:14.737504+0000","last_became_peered":"2024-09-15T21:30:14.737504+0000","last_unstale":"2024-09-15T21:30:52.354975+0000","last_undegraded":"2024-09-15T21:30:52.354975+0000","last_fullsized":"2024-09-15T21:30:52.354975+0000","mapping_epoch":65,"log_start":"46'8000","ondisk_log_start":"46'8000","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18034","last_scrub_stamp":"2024-09-15T21:30:52.354883+0000","last_deep_scrub":"54'18034","last_deep_scrub_stamp":"2024-09-15T21:30:52.354883+0000","last_clean_scrub_stamp":"2024-09-15T21:30:52.354883+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:54.817898+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15560,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3343,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"68'3","reported_seq":158,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:51.347537+0000","last_change":"2024-09-15T21:30:51.347537+0000","last_active":"2024-09-15T21:30:51.347537+0000","last_peered":"2024-09-15T21:30:51.347537+0000","last_clean":"2024-09-15T21:30:51.347537+0000","last_became_active":"2024-09-15T21:18:43.894437+0000","last_became_peered":"2024-09-15T21:18:43.894437+0000","last_unstale":"2024-09-15T21:30:51.347537+0000","last_undegraded":"2024-09-15T21:30:51.347537+0000","last_fullsized":"2024-09-15T21:30:51.347537+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'3","last_scrub_stamp":"2024-09-15T21:30:51.347474+0000","last_deep_scrub":"68'3","last_deep_scrub_stamp":"2024-09-15T21:30:51.347474+0000","last_clean_scrub_stamp":"2024-09-15T21:30:51.347474+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:41:59.536759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001848974,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"54'18285","reported_seq":24993,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:56.418910+0000","last_change":"2024-09-15T21:30:56.418910+0000","last_active":"2024-09-15T21:30:56.418910+0000","last_peered":"2024-09-15T21:30:56.418910+0000","last_clean":"2024-09-15T21:30:56.418910+0000","last_became_active":"2024-09-15T21:26:58.734893+0000","last_became_peered":"2024-09-15T21:26:58.734893+0000","last_unstale":"2024-09-15T21:30:56.418910+0000","last_undegraded":"2024-09-15T21:30:56.418910+0000","last_fullsized":"2024-09-15T21:30:56.418910+0000","mapping_epoch":53,"log_start":"46'8200","ondisk_log_start":"46'8200","created":17,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18285","last_scrub_stamp":"2024-09-15T21:30:56.418736+0000","last_deep_scrub":"54'18285","last_deep_scrub_stamp":"2024-09-15T21:30:56.418736+0000","last_clean_scrub_stamp":"2024-09-15T21:30:56.418736+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:41.316116+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16836,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"68'2","reported_seq":137,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.176271+0000","last_change":"2024-09-15T21:30:54.176066+0000","last_active":"2024-09-15T21:30:54.176271+0000","last_peered":"2024-09-15T21:30:54.176271+0000","last_clean":"2024-09-15T21:30:54.176271+0000","last_became_active":"2024-09-15T21:18:43.468702+0000","last_became_peered":"2024-09-15T21:18:43.468702+0000","last_unstale":"2024-09-15T21:30:54.176271+0000","last_undegraded":"2024-09-15T21:30:54.176271+0000","last_fullsized":"2024-09-15T21:30:54.176271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'2","last_scrub_stamp":"2024-09-15T21:30:54.175704+0000","last_deep_scrub":"68'2","last_deep_scrub_stamp":"2024-09-15T21:30:54.175704+0000","last_clean_scrub_stamp":"2024-09-15T21:30:54.175704+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:14.277639+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0014673749999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'18548","reported_seq":25431,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.433015+0000","last_change":"2024-09-15T21:30:54.433015+0000","last_active":"2024-09-15T21:30:54.433015+0000","last_peered":"2024-09-15T21:30:54.433015+0000","last_clean":"2024-09-15T21:30:54.433015+0000","last_became_active":"2024-09-15T21:28:20.732988+0000","last_became_peered":"2024-09-15T21:28:20.732988+0000","last_unstale":"2024-09-15T21:30:54.433015+0000","last_undegraded":"2024-09-15T21:30:54.433015+0000","last_fullsized":"2024-09-15T21:30:54.433015+0000","mapping_epoch":57,"log_start":"46'8500","ondisk_log_start":"46'8500","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18548","last_scrub_stamp":"2024-09-15T21:30:54.432815+0000","last_deep_scrub":"54'18548","last_deep_scrub_stamp":"2024-09-15T21:30:54.432815+0000","last_clean_scrub_stamp":"2024-09-15T21:30:54.432815+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:38.377979+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"54'18710","reported_seq":19500,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:31:01.245001+0000","last_change":"2024-09-15T21:31:01.244869+0000","last_active":"2024-09-15T21:31:01.245001+0000","last_peered":"2024-09-15T21:31:01.245001+0000","last_clean":"2024-09-15T21:31:01.245001+0000","last_became_active":"2024-09-15T21:28:47.901350+0000","last_became_peered":"2024-09-15T21:28:47.901350+0000","last_unstale":"2024-09-15T21:31:01.245001+0000","last_undegraded":"2024-09-15T21:31:01.245001+0000","last_fullsized":"2024-09-15T21:31:01.245001+0000","mapping_epoch":59,"log_start":"46'8700","ondisk_log_start":"46'8700","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18710","last_scrub_stamp":"2024-09-15T21:31:01.244686+0000","last_deep_scrub":"54'18710","last_deep_scrub_stamp":"2024-09-15T21:31:01.244686+0000","last_clean_scrub_stamp":"2024-09-15T21:31:01.244686+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:57.570235+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16017,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3434,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:56.188680+0000","last_change":"2024-09-15T21:30:56.188537+0000","last_active":"2024-09-15T21:30:56.188680+0000","last_peered":"2024-09-15T21:30:56.188680+0000","last_clean":"2024-09-15T21:30:56.188680+0000","last_became_active":"2024-09-15T21:18:43.605763+0000","last_became_peered":"2024-09-15T21:18:43.605763+0000","last_unstale":"2024-09-15T21:30:56.188680+0000","last_undegraded":"2024-09-15T21:30:56.188680+0000","last_fullsized":"2024-09-15T21:30:56.188680+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:56.188344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:56.188344+0000","last_clean_scrub_stamp":"2024-09-15T21:30:56.188344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:40.987812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002886511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"54'18412","reported_seq":25372,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:31:00.247265+0000","last_change":"2024-09-15T21:31:00.247265+0000","last_active":"2024-09-15T21:31:00.247265+0000","last_peered":"2024-09-15T21:31:00.247265+0000","last_clean":"2024-09-15T21:31:00.247265+0000","last_became_active":"2024-09-15T21:25:03.728468+0000","last_became_peered":"2024-09-15T21:25:03.728468+0000","last_unstale":"2024-09-15T21:31:00.247265+0000","last_undegraded":"2024-09-15T21:31:00.247265+0000","last_fullsized":"2024-09-15T21:31:00.247265+0000","mapping_epoch":51,"log_start":"46'8400","ondisk_log_start":"46'8400","created":17,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18412","last_scrub_stamp":"2024-09-15T21:31:00.247072+0000","last_deep_scrub":"54'18412","last_deep_scrub_stamp":"2024-09-15T21:31:00.247072+0000","last_clean_scrub_stamp":"2024-09-15T21:31:00.247072+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:18.281436+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15997,"num_write_kb":27180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3382,"num_bytes_recovered":3321856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:52.443670+0000","last_change":"2024-09-15T21:30:52.443670+0000","last_active":"2024-09-15T21:30:52.443670+0000","last_peered":"2024-09-15T21:30:52.443670+0000","last_clean":"2024-09-15T21:30:52.443670+0000","last_became_active":"2024-09-15T21:17:46.443516+0000","last_became_peered":"2024-09-15T21:17:46.443516+0000","last_unstale":"2024-09-15T21:30:52.443670+0000","last_undegraded":"2024-09-15T21:30:52.443670+0000","last_fullsized":"2024-09-15T21:30:52.443670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:52.443064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:52.443064+0000","last_clean_scrub_stamp":"2024-09-15T21:30:52.443064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:17:39.268223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"54'18378","reported_seq":19065,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:59.263753+0000","last_change":"2024-09-15T21:30:59.263753+0000","last_active":"2024-09-15T21:30:59.263753+0000","last_peered":"2024-09-15T21:30:59.263753+0000","last_clean":"2024-09-15T21:30:59.263753+0000","last_became_active":"2024-09-15T21:30:41.829132+0000","last_became_peered":"2024-09-15T21:30:41.829132+0000","last_unstale":"2024-09-15T21:30:59.263753+0000","last_undegraded":"2024-09-15T21:30:59.263753+0000","last_fullsized":"2024-09-15T21:30:59.263753+0000","mapping_epoch":67,"log_start":"46'8300","ondisk_log_start":"46'8300","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'18378","last_scrub_stamp":"2024-09-15T21:30:59.263571+0000","last_deep_scrub":"54'18378","last_deep_scrub_stamp":"2024-09-15T21:30:59.263571+0000","last_clean_scrub_stamp":"2024-09-15T21:30:59.263571+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:22.377561+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16025,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.185326+0000","last_change":"2024-09-15T21:30:55.185326+0000","last_active":"2024-09-15T21:30:55.185326+0000","last_peered":"2024-09-15T21:30:55.185326+0000","last_clean":"2024-09-15T21:30:55.185326+0000","last_became_active":"2024-09-15T21:18:43.606434+0000","last_became_peered":"2024-09-15T21:18:43.606434+0000","last_unstale":"2024-09-15T21:30:55.185326+0000","last_undegraded":"2024-09-15T21:30:55.185326+0000","last_fullsized":"2024-09-15T21:30:55.185326+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:55.185164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:55.185164+0000","last_clean_scrub_stamp":"2024-09-15T21:30:55.185164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:58.559335+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0024583209999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":130,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:52.147877+0000","last_change":"2024-09-15T21:30:52.147877+0000","last_active":"2024-09-15T21:30:52.147877+0000","last_peered":"2024-09-15T21:30:52.147877+0000","last_clean":"2024-09-15T21:30:52.147877+0000","last_became_active":"2024-09-15T21:18:43.381550+0000","last_became_peered":"2024-09-15T21:18:43.381550+0000","last_unstale":"2024-09-15T21:30:52.147877+0000","last_undegraded":"2024-09-15T21:30:52.147877+0000","last_fullsized":"2024-09-15T21:30:52.147877+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:30:52.147720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:30:52.147720+0000","last_clean_scrub_stamp":"2024-09-15T21:30:52.147720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:48.420126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027206930000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"68'18619","reported_seq":25760,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-15T21:30:57.267255+0000","last_change":"2024-09-15T21:30:57.267255+0000","last_active":"2024-09-15T21:30:57.267255+0000","last_peered":"2024-09-15T21:30:57.267255+0000","last_clean":"2024-09-15T21:30:57.267255+0000","last_became_active":"2024-09-15T21:22:00.564912+0000","last_became_peered":"2024-09-15T21:22:00.564912+0000","last_unstale":"2024-09-15T21:30:57.267255+0000","last_undegraded":"2024-09-15T21:30:57.267255+0000","last_fullsized":"2024-09-15T21:30:57.267255+0000","mapping_epoch":47,"log_start":"46'8600","ondisk_log_start":"46'8600","created":17,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'18619","last_scrub_stamp":"2024-09-15T21:30:57.267090+0000","last_deep_scrub":"68'18619","last_deep_scrub_stamp":"2024-09-15T21:30:57.267090+0000","last_clean_scrub_stamp":"2024-09-15T21:30:57.267090+0000","objects_scrubbed":1,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:44:32.421519+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":27900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3473,"num_bytes_recovered":14213120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267778,"num_write_kb":478545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40565,"num_bytes_recovered":44572672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":32,"seq":137438953642,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307128,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":94064712,"statfs":{"total":96636764160,"available":96322265088,"internally_reserved":0,"allocated":565248,"data_stored":189314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":26,"seq":111669149867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441728,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":441147,"kb_avail":93930112,"statfs":{"total":96636764160,"available":96184434688,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4431,"internal_metadata":451735217},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":42,"seq":180388626586,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473728,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898112,"statfs":{"total":96636764160,"available":96151666688,"internally_reserved":0,"allocated":573440,"data_stored":189950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4436,"internal_metadata":484503212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:12.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:31:13.274 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:31:13.275 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:31:13.275 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:31:13.275 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:18.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T21:31:19.377 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:31:19.377 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:31:19.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:19.469 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:31:19.469 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:31:19.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:23.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T21:31:23.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-15T21:31:25.571 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:31:25.571 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:31:25.572 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:31:25.572 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:25.601 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:31:25.602 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:31:25.602 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:31:25.602 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:25.657 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:31:25.657 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:31:25.657 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:25.716 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:31:25.717 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:31:25.717 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:25.777 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:31:25.777 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:31:25.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-15T21:31:25.873 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:31:25.873 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:31:26.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:31:26.133 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:31:26.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:31:26.365 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:31:26.558 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:31:26.559 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/data/mon.a.tgz 2024-09-15T21:31:26.559 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-15T21:31:26.579 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.oewlwkG3lz 2024-09-15T21:31:26.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oewlwkG3lz 2024-09-15T21:31:26.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.oewlwkG3lz 2024-09-15T21:31:26.797 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.oewlwkG3lz is 317KB 2024-09-15T21:31:26.858 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.oewlwkG3lz 2024-09-15T21:31:26.870 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/data/mon.b.tgz 2024-09-15T21:31:26.870 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-15T21:31:26.917 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.EhAXU1fzuC 2024-09-15T21:31:26.918 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EhAXU1fzuC 2024-09-15T21:31:27.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.EhAXU1fzuC 2024-09-15T21:31:27.105 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.EhAXU1fzuC is 329KB 2024-09-15T21:31:27.166 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.EhAXU1fzuC 2024-09-15T21:31:27.172 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/data/mon.c.tgz 2024-09-15T21:31:27.172 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-15T21:31:27.221 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.8MAA3Joyth 2024-09-15T21:31:27.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8MAA3Joyth 2024-09-15T21:31:27.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.8MAA3Joyth 2024-09-15T21:31:27.413 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.8MAA3Joyth is 329KB 2024-09-15T21:31:27.478 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.8MAA3Joyth 2024-09-15T21:31:27.484 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:31:27.484 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:31:27.620 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:31:27.620 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:31:27.621 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/remote/smithi114/crash 2024-09-15T21:31:27.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:31:27.677 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:31:27.677 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:31:27.744 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29774.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43473.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29774.log.gz 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43473.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43473.log.gz 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37065.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38339.log 2024-09-15T21:31:28.012 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37065.log: /var/log/ceph/ceph-client.admin.38339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42875.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38339.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37065.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30640.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42875.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43161.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30640.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43161.log.gz 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31745.log 2024-09-15T21:31:28.013 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31745.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33437.log 2024-09-15T21:31:28.014 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29796.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33437.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29796.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48186.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38391.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41542.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48186.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38391.log.gz 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-09-15T21:31:28.015 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42927.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34477.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28539.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31663.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41829.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34552.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38599.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30038.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31975.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31692.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28389.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33801.log 2024-09-15T21:31:28.016 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41360.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44488.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32893.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35080.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42745.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44124.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30016.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38027.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39537.log 2024-09-15T21:31:28.017 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44384.log 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41542.log: /var/log/ceph/ceph-client.admin.49512.log: /var/log/ceph/ceph-client.admin.39459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41542.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39877.log: /var/log/ceph/ceph-client.admin.42927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42927.log.gz -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34477.log: /var/log/ceph/ceph-client.admin.28539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44384.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31663.log: /var/log/ceph/ceph-client.admin.41829.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz -- replaced with /var/log/ceph/ceph-client.admin.28539.log.gz 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.018 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31663.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41829.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38599.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31975.log: /var/log/ceph/ceph-client.admin.31692.log: /var/log/ceph/ceph-client.admin.28389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% 0.0%/var/log/ceph/ceph-client.admin.34552.log: -- replaced with /var/log/ceph/ceph-client.admin.31975.log.gz -- replaced with /var/log/ceph/ceph-client.admin.30038.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31692.log.gz -- replaced with /var/log/ceph/ceph-client.admin.28389.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz/var/log/ceph/ceph-client.admin.33801.log: 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41360.log: /var/log/ceph/ceph-client.admin.44488.log: /var/log/ceph/ceph-client.admin.32893.log: /var/log/ceph/ceph-client.admin.35080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33801.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44488.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35080.log.gz -- replaced with /var/log/ceph/ceph-client.admin.32893.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41360.log.gz 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.019 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45428.log: /var/log/ceph/ceph-client.admin.49770.log: /var/log/ceph/ceph-client.admin.38313.log: /var/log/ceph/ceph-client.admin.33047.log: /var/log/ceph/ceph-client.admin.42745.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.46806.log: 0.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42745.log.gz /var/log/ceph/ceph-client.admin.38782.log 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44124.log: /var/log/ceph/ceph-client.admin.30016.log: /var/log/ceph/ceph-client.admin.38027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30016.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44124.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38027.log.gz 2024-09-15T21:31:28.020 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45818.log: /var/log/ceph/ceph-client.admin.39537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gzgzip 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39537.log.gz 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47353.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40397.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47353.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47353.log.gz -- 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.39981.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40397.log: -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40397.log.gz 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39981.log.gz 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47119.log 2024-09-15T21:31:28.021 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48082.log: /var/log/ceph/ceph-client.admin.47119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49590.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47119.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30288.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49590.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42719.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30288.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.39355.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42953.log 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42719.log.gz 2024-09-15T21:31:28.022 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29752.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42953.log: 29.2% -- replaced with /var/log/ceph/ceph-client.admin.34552.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42953.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29752.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47535.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.43499.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30684.log 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-09-15T21:31:28.023 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43915.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30684.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43915.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45896.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29344.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45896.log.gz 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46182.log 2024-09-15T21:31:28.024 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33286.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29344.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36337.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43265.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33286.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36623.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36337.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33827.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43265.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46000.log 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36623.log.gz 2024-09-15T21:31:28.025 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28780.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33827.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46000.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28780.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.38887.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40163.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39277.log 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2024-09-15T21:31:28.026 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29972.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40163.log.gz 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46286.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39277.log.gz 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29972.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.29972.log.gz --verbose 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: -- /var/log/ceph/ceph-client.admin.29244.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.46286.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28851.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46286.log.gz 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30082.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29244.log.gz 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28259.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28851.log.gz 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28617.log 2024-09-15T21:31:28.027 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30082.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30104.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28259.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28617.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28617.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34061.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30104.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34061.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47275.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-09-15T21:31:28.028 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47275.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28357.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33567.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28357.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42797.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46104.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33567.log.gz 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36805.log 2024-09-15T21:31:28.029 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30508.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46104.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36805.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39929.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30508.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30060.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38547.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39929.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30060.log.gz 2024-09-15T21:31:28.030 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38547.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28331.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38235.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40475.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28331.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41204.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38235.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 29.8% -- replaced with /var/log/ceph/ceph-client.admin.40475.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41204.log.gz 2024-09-15T21:31:28.031 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47093.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41985.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44829.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47093.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39721.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41985.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28721.log 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz 2024-09-15T21:31:28.032 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44176.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39721.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34794.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28721.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30728.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44176.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34794.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41568.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30728.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44019.log 2024-09-15T21:31:28.033 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41568.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42141.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25160.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44019.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34684.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30772.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25160.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29240.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34684.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49538.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30772.log.gz 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45220.log 2024-09-15T21:31:28.034 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29240.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49538.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28643.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45220.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28643.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48758.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30310.log 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48758.log.gz 2024-09-15T21:31:28.035 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28469.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28695.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30310.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37715.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28469.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28695.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39799.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37715.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28937.log 2024-09-15T21:31:28.036 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39799.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42508.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37455.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28937.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42508.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37455.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46052.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz 2024-09-15T21:31:28.037 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46052.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.30244.log -- replaced with /var/log/ceph/ceph-client.admin.46052.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40605.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42427.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30244.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48108.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40605.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34247.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42427.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34926.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.48108.log.gz 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40995.log 2024-09-15T21:31:28.038 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34247.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44960.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34926.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40995.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38209.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44960.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.38209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38209.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46026.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43551.log 2024-09-15T21:31:28.039 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30706.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46026.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29708.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43551.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29531.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30706.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29598.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29708.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29531.log.gz 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31797.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-09-15T21:31:28.040 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29598.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34574.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31797.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30530.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35036.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34574.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30530.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34750.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35036.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41620.log 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-09-15T21:31:28.041 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38157.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34750.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46208.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41620.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33775.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38157.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33235.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46208.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36675.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33775.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40813.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33235.log.gz 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-09-15T21:31:28.042 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36675.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40813.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48498.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41594.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28778.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45922.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41594.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28778.log.gz 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-09-15T21:31:28.043 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45922.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38730.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40371.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38730.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41073.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37403.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40371.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41073.log.gz 2024-09-15T21:31:28.044 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37403.log.gz 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33515.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33515.log.gz 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37429.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37429.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.37429.log.gz -- 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.43395.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34503.log 2024-09-15T21:31:28.045 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36311.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33075.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34503.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29477.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36311.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45480.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33075.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29477.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45480.log.gz 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29686.log 2024-09-15T21:31:28.046 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32867.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48056.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29686.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29059.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.32867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37013.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32867.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48056.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29059.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37013.log.gz 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-09-15T21:31:28.047 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45142.log: -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40345.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29007.log 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40345.log.gz 2024-09-15T21:31:28.048 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.29007.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41777.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29007.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42823.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41777.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42823.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-09-15T21:31:28.049 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-09-15T21:31:28.050 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-09-15T21:31:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-09-15T21:31:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-09-15T21:31:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38756.log 2024-09-15T21:31:28.050 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30596.log 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40735.log 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38756.log.gz 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47067.log 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30596.log.gz 2024-09-15T21:31:28.052 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40735.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40033.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46572.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40033.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46572.log.gz 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2024-09-15T21:31:28.053 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42297.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37923.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47717.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42297.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39851.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37923.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47717.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30126.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39851.log.gz 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-09-15T21:31:28.054 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30126.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36909.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43993.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36883.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36909.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49304.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37169.log 2024-09-15T21:31:28.055 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36883.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49304.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37169.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35014.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29088.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35014.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-09-15T21:31:28.056 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29088.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28390.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34662.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40189.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28390.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38625.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34662.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31693.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40189.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz 2024-09-15T21:31:28.057 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31693.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36207.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44332.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 29.2% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36207.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46364.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44332.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40267.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46364.log.gz 2024-09-15T21:31:28.058 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40267.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46520.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33905.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46520.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48550.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35058.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33905.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43031.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48550.log.gz 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43083.log 2024-09-15T21:31:28.059 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28859.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43031.log: 29.3% -- replaced with /var/log/ceph/ceph-client.admin.35058.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43031.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43083.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29425.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28859.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39303.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36129.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29425.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46624.log 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39303.log.gz 2024-09-15T21:31:28.060 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36129.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43733.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46624.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43733.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30177.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33671.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30177.log.gz 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40059.log 2024-09-15T21:31:28.061 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36363.log/var/log/ceph/ceph-client.admin.33671.log: 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33671.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34194.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40059.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33541.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36363.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34194.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49564.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-09-15T21:31:28.062 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49564.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37871.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28747.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37871.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29884.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28747.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36155.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29884.log.gz 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46260.log 2024-09-15T21:31:28.063 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36155.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30266.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46260.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30266.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30750.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33153.log 2024-09-15T21:31:28.064 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30354.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30750.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45090.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33153.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30354.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47197.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45090.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44540.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47197.log.gz 2024-09-15T21:31:28.065 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33127.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44540.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33127.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33127.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.40969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28305.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44751.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-09-15T21:31:28.066 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28305.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42560.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48420.log: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45974.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42560.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43837.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45974.log.gz 2024-09-15T21:31:28.067 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47795.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43837.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38443.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49382.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47795.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37819.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38443.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49382.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39485.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37819.log.gz 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37975.log 2024-09-15T21:31:28.068 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32974.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39485.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37897.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37975.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.32974.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46598.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32974.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37897.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28781.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46598.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-09-15T21:31:28.069 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28781.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33359.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36233.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49200.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41230.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36233.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44934.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49200.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39173.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41230.log.gz 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48342.log 2024-09-15T21:31:28.070 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44934.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44934.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.39173.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39173.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48316.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48342.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37247.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48316.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28546.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37247.log.gz 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41412.log 2024-09-15T21:31:28.071 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28546.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28858.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41412.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29642.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34618.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28858.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44986.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29642.log.gz 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2024-09-15T21:31:28.072 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34618.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34087.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44986.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41438.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34087.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41438.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40215.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46546.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40215.log: /var/log/ceph/ceph-client.admin.35102.log 2024-09-15T21:31:28.073 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40215.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41933.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46546.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41490.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35102.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31741.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41933.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39565.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41490.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31741.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44595.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31793.log 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39565.log.gz 2024-09-15T21:31:28.074 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33931.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44595.log.gz 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31793.log.gz 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33931.log.gz 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz /var/log/ceph/ceph-client.admin.36441.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44410.log 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-09-15T21:31:28.075 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36441.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37039.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44410.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39225.log: 29.2% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40631.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37039.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42375.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47171.log 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40631.log.gz 2024-09-15T21:31:28.076 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47171.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28234.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33957.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28234.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43967.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33957.log.gz 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41256.log 2024-09-15T21:31:28.077 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43967.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36961.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33385.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41256.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48160.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36961.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36415.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33385.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39511.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48160.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39199.log 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36415.log.gz 2024-09-15T21:31:28.078 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31923.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39511.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30574.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39199.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30151.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31923.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29818.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30574.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30151.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37663.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29818.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42771.log 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2024-09-15T21:31:28.079 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37663.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37663.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36649.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42771.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36545.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29928.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36649.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49174.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36545.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29576.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29928.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49174.log.gz 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29033.log 2024-09-15T21:31:28.080 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29576.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25509.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29033.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40449.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40657.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25509.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44514.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40657.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40657.log.gz --verbose 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: -- /var/log/ceph/ceph-client.admin.34244.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz 2024-09-15T21:31:28.081 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44699.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34244.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25385.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46650.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44699.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25385.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46650.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37325.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43343.log 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-09-15T21:31:28.082 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40683.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37325.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46780.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43343.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37533.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40683.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48940.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46780.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36831.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37533.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30420.log 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz 2024-09-15T21:31:28.083 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36831.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30420.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-09-15T21:31:28.084 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31949.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33879.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31949.log.gz 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44621.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49278.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33879.log.gz 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29730.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44621.log.gz 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37741.log 2024-09-15T21:31:28.085 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49278.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33182.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29730.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29296.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37741.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30662.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33182.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33619.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29296.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37611.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30662.log.gz 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37143.log 2024-09-15T21:31:28.086 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33619.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37611.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49252.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37143.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40787.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43629.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49252.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49642.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40787.log.gz 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41178.log 2024-09-15T21:31:28.087 INFO:teuthology.orchestra.run.smithi114.stderr: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.43629.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49642.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41646.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41178.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41646.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr: /var/log/ceph/ceph-client.admin.42219.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45844.log 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-15T21:31:28.088 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37273.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 30.2% -- replaced with /var/log/ceph/ceph-client.admin.45844.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37273.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29140.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45948.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30398.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29140.log.gz 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38808.log 2024-09-15T21:31:28.089 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45948.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41725.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30398.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44280.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38808.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44045.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41725.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44280.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44045.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44045.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48472.log 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-09-15T21:31:28.090 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47587.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.48472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28048.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48472.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30552.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47587.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44306.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28048.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30552.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44306.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29554.log 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-09-15T21:31:28.091 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37351.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29554.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43317.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28669.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37351.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37559.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43317.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47952.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28669.log.gz 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-09-15T21:31:28.092 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37559.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47952.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44725.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33489.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44725.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32923.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33489.log.gz 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-09-15T21:31:28.093 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.32923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39433.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32923.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33335.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39433.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33335.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-15T21:31:28.094 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34596.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25306.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34882.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34596.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36493.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25306.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33183.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34882.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36493.log.gz 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31845.log 2024-09-15T21:31:28.095 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33183.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31845.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36779.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45038.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36779.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34992.log 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-09-15T21:31:28.096 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25409.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45038.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47041.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34992.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25409.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40527.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47041.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29292.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40527.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44071.log 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-09-15T21:31:28.097 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29292.log.gz 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39043.log/var/log/ceph/ceph-client.admin.44071.log: 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44071.log.gz 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32975.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39043.log.gz 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.32975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30332.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29994.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48628.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30332.log.gz 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29994.log: /var/log/ceph/ceph-client.admin.48628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29840.log 2024-09-15T21:31:28.098 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48628.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37221.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34970.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29840.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41386.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37221.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38495.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30464.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41386.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41881.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38495.log.gz 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36857.log 2024-09-15T21:31:28.099 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30464.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25361.log 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41881.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40137.log 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36857.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25361.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.29994.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40137.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47015.log 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-09-15T21:31:28.100 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36987.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.47015.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47015.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28461.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36181.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28461.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41751.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36181.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44673.log 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-09-15T21:31:28.101 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41751.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44673.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43811.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29451.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43811.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36467.log/var/log/ceph/ceph-client.admin.46468.log: 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46156.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29451.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36467.log.gz 2024-09-15T21:31:28.102 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46156.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30376.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34816.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30376.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33023.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34816.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2024-09-15T21:31:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29396.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33023.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21546.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29396.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33411.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.tmp-client.admin.21546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48654.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21546.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28468.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33411.log.gz 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49720.log 2024-09-15T21:31:28.104 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48654.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36701.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28468.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49720.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36701.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.35124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29906.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29503.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47561.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29906.log.gz 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30618.log 2024-09-15T21:31:28.105 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29503.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32001.log 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47561.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38001.log 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30618.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.32001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31637.log 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32001.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39955.log 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38001.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31637.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39955.log.gz 2024-09-15T21:31:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34772.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46885.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42011.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34772.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46885.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46885.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37845.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33853.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42011.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37507.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29114.log 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33853.log.gz 2024-09-15T21:31:28.107 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38183.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37507.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45506.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29114.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46494.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38183.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34948.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45506.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46494.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41334.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34948.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2024-09-15T21:31:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41334.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28933.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28933.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.28933.log.gz --verbose 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr: -- /var/log/ceph/ceph-client.admin.33723.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28547.log 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33723.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2024-09-15T21:31:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28547.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30486.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38521.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.30486.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41464.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30486.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39069.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 29.8% -- replaced with /var/log/ceph/ceph-client.admin.38521.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29191.log 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41464.log.gz 2024-09-15T21:31:28.110 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39069.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29192.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29191.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41516.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29192.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.29192.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40085.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40007.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41516.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34009.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40085.log.gz 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-09-15T21:31:28.111 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40007.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37091.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34009.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.45350.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31849.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42641.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37091.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38573.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31849.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.42641.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41282.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42641.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29348.log 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38573.log.gz 2024-09-15T21:31:28.112 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48602.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41282.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33593.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29348.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43005.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48602.log: /var/log/ceph/ceph-client.admin.33593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48602.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33593.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38469.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43005.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37377.log 2024-09-15T21:31:28.113 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38469.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47327.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37377.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37377.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33234.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47327.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43109.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36285.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33234.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49460.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43109.log.gz 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-09-15T21:31:28.114 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36285.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38079.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49460.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44777.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36519.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38079.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41803.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44777.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36519.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41803.log.gz 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36727.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2024-09-15T21:31:28.115 INFO:teuthology.orchestra.run.smithi114.stderr: 29.2% -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36727.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43655.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29664.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43655.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37949.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29664.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48680.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48706.log 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37949.log.gz 2024-09-15T21:31:28.116 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46078.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48680.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34195.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48706.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46078.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38704.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34195.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38704.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25444.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46832.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.25444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-09-15T21:31:28.117 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25444.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37585.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46832.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36935.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47639.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36389.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36935.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43577.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47639.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36753.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36389.log.gz 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41907.log 2024-09-15T21:31:28.118 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43577.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30222.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36753.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41152.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41907.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38678.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30222.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29950.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41152.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43681.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38678.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29950.log.gz 2024-09-15T21:31:28.119 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37689.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43681.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.37689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28279.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37689.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36597.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33697.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28279.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36597.log.gz 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47743.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34838.log 2024-09-15T21:31:28.120 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33697.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41308.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47743.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34838.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28391.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.41308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41308.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34299.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28391.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38053.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33463.log 2024-09-15T21:31:28.121 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34299.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38053.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39407.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33463.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49018.log: 28.3% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39407.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34035.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.48264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.34035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2024-09-15T21:31:28.122 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34035.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32922.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42667.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49616.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.32922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32922.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42667.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38834.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49616.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44462.log 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-09-15T21:31:28.123 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47301.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38834.log.gz 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28209.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44462.log.gz 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.28209.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.28209.log.gz /var/log/ceph/ceph-client.admin.29620.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38105.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29620.log.gz 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33749.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.38105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2024-09-15T21:31:28.124 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38105.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33749.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41855.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42534.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:gzip/var/log/ceph/ceph-client.admin.41855.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30200.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41855.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42534.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30200.log.gz 2024-09-15T21:31:28.125 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40709.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42349.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.46338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39147.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40709.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42349.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47431.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39147.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.47431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29862.log 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr: 29.9% -- replaced with /var/log/ceph/ceph-client.admin.47431.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.49668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-09-15T21:31:28.126 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.29862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29862.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33101.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.45012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36259.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30442.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33101.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.36259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40111.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36259.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39095.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.30442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30442.log.gz 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31897.log 2024-09-15T21:31:28.127 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40111.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.39095.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39095.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.31897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29163.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31897.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.42193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5/var/log/ceph/ceph-client.admin.29163.log: --verbose -- /var/log/ceph/ceph-client.admin.45584.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29163.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.32975.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.43213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44150.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5/var/log/ceph/ceph-client.admin.45584.log: --verbose -- /var/log/ceph/ceph-client.admin.40579.log 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45584.log.gz 2024-09-15T21:31:28.128 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.44150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44150.log.gz 2024-09-15T21:31:28.129 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.40579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40579.log.gz 2024-09-15T21:31:28.129 INFO:teuthology.orchestra.run.smithi114.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.33359.log.gz 2024-09-15T21:31:28.129 INFO:teuthology.orchestra.run.smithi114.stderr: 96.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:31:28.197 INFO:teuthology.orchestra.run.smithi114.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:31:28.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-15T21:31:28.933 INFO:teuthology.orchestra.run.smithi114.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:31:29.213 INFO:teuthology.orchestra.run.smithi114.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:31:29.695 INFO:teuthology.orchestra.run.smithi114.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:32:51.763 INFO:teuthology.orchestra.run.smithi114.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:35:00.166 INFO:teuthology.orchestra.run.smithi114.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:35:00.166 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-15T21:35:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:real 3m32.440s 2024-09-15T21:35:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:user 5m29.531s 2024-09-15T21:35:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m12.221s 2024-09-15T21:35:00.167 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:35:00.167 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/remote/smithi114/log 2024-09-15T21:35:00.168 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:35:43.144 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:35:43.158 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:35:43.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:35:43.197 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:35:43.209 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:35:43.210 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:35:43.334 INFO:teuthology.orchestra.run.smithi114.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T21:35:43.335 INFO:teuthology.orchestra.run.smithi114.stdout:============================================================================== 2024-09-15T21:35:43.335 INFO:teuthology.orchestra.run.smithi114.stdout:+hv01.front.sepi 67.205.162.81 3 u 39 64 377 0.087 -5.727 0.855 2024-09-15T21:35:43.335 INFO:teuthology.orchestra.run.smithi114.stdout:*hv02.front.sepi 63.231.80.2 3 u 30 64 377 0.081 -4.987 0.284 2024-09-15T21:35:43.335 INFO:teuthology.orchestra.run.smithi114.stdout:+hv03.front.sepi 74.6.168.72 3 u 34 64 377 0.091 -3.046 0.757 2024-09-15T21:35:43.335 INFO:teuthology.orchestra.run.smithi114.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-15T21:35:43.336 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:35:43.347 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:35:43.348 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:35:43.374 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:35:43.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:35:43.416 INFO:teuthology.task.internal:Duration was 1737.489726 seconds 2024-09-15T21:35:43.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:35:43.424 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:35:43.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:35:43.590 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:35:43.590 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-15T21:35:43.591 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:35:43.601 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:35:43.601 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:35:43.673 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:35:43.673 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:35:43.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:35:43.892 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:35:43.892 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:35:43.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:35:43.926 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:35:43.983 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-15T21:35:44.019 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:35:44.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:35:44.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:35:44.077 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:35:44.077 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905674/remote/smithi114 2024-09-15T21:35:44.078 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:35:44.132 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:35:44.132 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:35:44.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:35:44.186 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:35:44.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:35:44.213 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:35:44.213 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:35:44.224 INFO:teuthology.orchestra.run.smithi114.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:35 /home/ubuntu/cephtest 2024-09-15T21:35:44.226 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:35:44.236 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:35:44.257 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zlib} rados supported-random-distro$/{ubuntu_latest}} duration: 1737.4897258281708 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:35:44.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:35:44.341 INFO:teuthology.run:pass